This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

c6678 Packet Accelerator - doing IP and UDP checksum on outgoing packets - Need help?

Hi TI folks,

Attn: Eric or any other designer in the PA or multicore navigator queue etc

I am trying to get ethernet packets sent out through the switch on the c6678. So I am using the 9th port (which is the switch transmit port) when I push a descriptor with the ethernet packet location specified in the buffer location in the descriptor this works and I am able to capture the packets sent by the EVM. However, I have the IP and UDP checksums set to zero which is obviously incorrect. I now want the PDSP4 to compute the checksums and then have the packet routed to the transmit queue of the switch so I instead push the packets onto the 5th port (which is PDSP4). I am making use of the Pa_formatTxRoute command where I can specify the IPv4 header checksum as NULL, UDP as null and then set up the route and the command buffer and size  and this too works as the size of the command is only 12 bytes long. Please note my TX descriptors are 64 bytes long, 32bytes of regular data plus 16 bytes of software info etc plus 16 bytes of PS data. So as the PS data is less than 16 bytes, this case works and I am able to capture the outgoing packets. However when I add the pktIpChksum command, the Pa_formatTxRoute returns a command that is larger than 16 bytes (24 bytes to be precise) it obviously cannot fit in the PS section overwriting the next descriptor. Now I could make this TX descriptors much larger like 96 or 128 bytes long but I am hoping I could make use of a suggestion in the multicore navigator user guide under programming consideration TIp2 which I have pasted below.

TIP2:
You must program a descriptor region with a fixed size, but you do not have to use every descriptor. As long as each descriptor is a multiple of the
programmed size (which itself is a multiple of 16 bytes), you can use contiguousdescriptors to create a single larger descriptor. The host must manage how it tracks the different sized descriptors.

I am also including my SendPacket function which is based on the SendPacket that is part of the multicore PA example project that comes with PDK. How can I go about making use of two descriptors as one. Some programming suggested steps would be useful i.e. Do I have to pop 2 descriptors etc. Alternatively can I have the PS information stored in the buffers pointed to by the descriptors i.e PD Word 4 in the descriptor as specified in the navigator user guide and does this ensure that the PS info is actually not transmitted as part of the packet but rather used by the PDSP4 to perform the checksum computation before routing to the switch for transmission.

Thanks, Aamir

--------------------------------------------------------------------------------------------

Int32 SendPacket (unsigned char*  pucOutData, UInt32 dataBufferSize)
{
    Cppi_HostDesc*  pCppiDesc;
    uint32_t        key;
    char            psFlags = 0x1;
    uint32_t     cmdStack[(sizeof(pasahoNextRoute_t) + (2 * sizeof(pasahoComChkCrc_t))) / sizeof (uint32_t)];
    UInt16          cmdSize = CMD_BUF_SIZE;
    paRouteInfo_t  route =  {  pa_DEST_EMAC,  /* Route - host         */
                               0,             /* flow Id              */
                               0,       /* Queue                */
                               -1,            /* Multi route disabled */
                               0,             /* SWInfo 0             */
                               0,             /* SWInfo 1 */      
                               0,             /* customType : not used */        
                               0,             /* customIndex: not used */    
                               0,             /* pkyType: for SRIO only */   
                               NULL};         /* No commands */
    paReturn_t                  retVal;
                              
   
 //
    retVal = Pa_formatTxRoute (
           &pktIpChksum,      /* IPv4 header checksum */
                                NULL,                /* No second checksum   */
                                &route,              /* Internal routing     */
                                (Ptr)(pasahoNextRoute_t *)cmdStack,           /* Command buffer       */
                                &cmdSize);   /* Command size         */


    if (retVal != pa_OK)  {
        System_printf ("Pa_formatTxRoute returned error code %d\n", retVal);
        return -1;
    }

    /* Get a free descriptor from the global free queue we setup
     * during initialization.
     */
    if ((pCppiDesc = Qmss_queuePop (gTxFreeQHnd)) == NULL)
    {
        System_printf ("No Tx free descriptor. Cant run send/rcv test \n");
        return -1;
    }
   
    /* The descriptor address returned from the hardware has the
     * descriptor size appended to the address in the last 4 bits.
     *
     * To get the true descriptor size, always mask off the last
     * 4 bits of the address.
     */
    pCppiDesc = (Ptr) ((UInt32) pCppiDesc & 0xFFFFFFF0);
   
    /* Disable Interrupts */
    key = Hwi_disable();

    /* Cleanup the prefetch buffer also. */
    CSL_XMC_invalidatePrefetchBuffer();
   
    SYS_CACHE_INV (pCppiDesc, SIZE_HOST_DESC, CACHE_FENCE_WAIT);
   
    Cppi_setData (  Cppi_DescType_HOST,
                    (Cppi_Desc *) pCppiDesc,
                    //(UInt8 *) Convert_CoreLocal2GlobalAddr((UInt32)pktMatch),
                    (UInt8 *) Convert_CoreLocal2GlobalAddr((UInt32)pucOutData),
                    dataBufferSize
                 );
    Cppi_setPacketLen (Cppi_DescType_HOST, (Cppi_Desc *)pCppiDesc, dataBufferSize);
   
    if (cpswLpbkMode != CPSW_LOOPBACK_NONE)
    {
        /* Force the packet to EMAC port 0 if loopback is enabled */
        Cppi_setPSFlags(Cppi_DescType_HOST, (Cppi_Desc *)pCppiDesc, psFlags);
    }
   
    /* Clear PS Data */
    Cppi_setPSLen (Cppi_DescType_HOST, (Cppi_Desc *)pCppiDesc, 0);
 
    /* Attach the command in PS data */
    Cppi_setPSData (Cppi_DescType_HOST, (Cppi_Desc *)pCppiDesc, (uint8_t *)cmdStack, cmdSize);

    SYS_CACHE_WB (pCppiDesc, SIZE_HOST_DESC, CACHE_FENCE_WAIT);
 
    /* Reenable Interrupts. */
    Hwi_restore(key);
 
    /* Send the packet out the mac. It will loop back to PA if the mac/switch
     * have been configured properly
     */ 
    //Qmss_queuePushDescSize (gPaTxQHnd[8], pCppiDesc, SIZE_HOST_DESC);
    Qmss_queuePushDescSize (gPaTxQHnd[4], pCppiDesc, SIZE_HOST_DESC);

    /* Increment the application transmit counter */
    gTxCounter ++;

    /* Give some time for the PA to process the packet */
    CycleDelay (10000);

    return 0;
}

  • Hi, Aamir:

    Yes, you can put the psInfo data at the SOP packet. You may call Cppi_setPSLocation() to sepcify the PS location. If you want to do the same in the rx direction, then you need to configure rx_ps_location of the CPPI flow.

    If you want to keep the psInfo at the CPPI descriptor, you need to set the descriptor size to 80byte at minimum, you way want to change it to 128-byte for cache alignment.

    Best regards,

    Eric

     

     

     

     

     

  • Eric,

    Good to hear from you. Thanks for the advice. Since I last posted here is what I managed to try.

    I tried modifying the size I push on the queue from 64 to 128 by modifying Qmss_queuePushDescSize. When I do that I am able to get the packet out of the DSP and capture the packet and check that the IPv4 headerchecksum matches.  Also in the Pa_formatTxRoute command it has space for two checksum info to be passed to the function, can I just use them both and specify one as pktIpChksum and the other as pktUpdChksum and then I do not need to do the pa_formatTxCmd? I tried that and was not able to get that to work - i.e. no packets came out. I also gather that for the case of UDP checksum, I actually have to pass in the checksum for the IPV4 psuedo header as I tried it by putting zero for the UDP checksum only and I was out by the pseudo header in the checksum computation. I then manually computed the IPv4 pseudo header checksum and initialized it to that and then the received packet in my capture had its UDP checksum match. Another thing I discovered is that if I put a breakpoint in the Send function at the pa_formatTxRoute() and manually step through the code till the point where the packet is actually pushed out to the EMAC it works but if I put a breakpoint at the point where it is pushed out instead then it does not work and I see no packets going out of the DSP- That seems very strange - Any explanation? Note that I have code that triggers an outgoing packet whenever an incoming packet is received by the host.

    I will try putting the PS info prior to my packet data as I really do not want to change my descriptors to be more than 64 bytes long. I plan on having 1000's of these so it will be rather wasteful. With the two checksum computations and the route, I believe I need 12*3=36 bytes of PS info. I thought I could use my intended descriptor and the one next to it as one descriptor but that is a little complicated (this is tip2 in the user guide). Do you suggest that option is even workable?

  • Eric,

    I made changes to my sendPacket function to put the PS info prior to the SOP by doing the setPsLocation as shown below. I did this prior to setting the data etc. I just made some more space for the 36 bytes extra for PS info prior to the ethernet packet but when I do the Cppi_setPSData with my command to be attached to the PS data it does not get placed before the ethernet packet.

    Cppi_setPSLocation (Cppi_DescType_HOST, (Cppi_Desc *)pCppiDesc, Cppi_PSLoc_PS_IN_SOP);

    Thanks, Aamir

     

  • Hi, Aamir:

    Please see my answers in-line:

    I tried modifying the size I push on the queue from 64 to 128 by modifying Qmss_queuePushDescSize. When I do that I am able to get the packet out of the DSP and capture the packet and check that the IPv4 headerchecksum matches.  Also in the Pa_formatTxRoute command it has space for two checksum info to be passed to the function, can I just use them both and specify one as pktIpChksum and the other as pktUpdChksum and then I do not need to do the pa_formatTxCmd? I tried that and was not able to get that to work - i.e. no packets came out.

    [Eric] It should work. Can you send me the psInfo dump before you push the packet into the queue.

    I also gather that for the case of UDP checksum, I actually have to pass in the checksum for the IPV4 psuedo header as I tried it by putting zero for the UDP checksum only and I was out by the pseudo header in the checksum computation. I then manually computed the IPv4 pseudo header checksum and initialized it to that and then the received packet in my capture had its UDP checksum match.

    [Eric] Yes, you do need to provide IP psudo checksum, there is no protocol awareness in the UDP/IP checksum. The Checksum IP will just sum up all 16-bit words within the checksum range.

    Another thing I discovered is that if I put a breakpoint in the Send function at the pa_formatTxRoute() and manually step through the code till the point where the packet is actually pushed out to the EMAC it works but if I put a breakpoint at the point where it is pushed out instead then it does not work and I see no packets going out of the DSP- That seems very strange - Any explanation?

    [Eric] You may have a cache or buffering problem. Can you send me the psInfo dump prior to packet pushing in both cases.

    Note that I have code that triggers an outgoing packet whenever an incoming packet is received by the host.

    I thought I could use my intended descriptor and the one next to it as one descriptor but that is a little complicated (this is tip2 in the user guide). Do you suggest that option is even workable?

    [Eric]  No, I do not think that it is a good idea since you are going to overwrite the adjacent CPPI descriptor and how do you plan to recover them at the end.

    Best regards,

    Eric

     

     

  • Hi, Aamir:

    Please see my answers in-line:

    I tried modifying the size I push on the queue from 64 to 128 by modifying Qmss_queuePushDescSize. When I do that I am able to get the packet out of the DSP and capture the packet and check that the IPv4 headerchecksum matches.  Also in the Pa_formatTxRoute command it has space for two checksum info to be passed to the function, can I just use them both and specify one as pktIpChksum and the other as pktUpdChksum and then I do not need to do the pa_formatTxCmd? I tried that and was not able to get that to work - i.e. no packets came out.

    [Eric] It should work. Can you send me the psInfo dump before you push the packet into the queue.

    [Aamir] It does work now for some reason. Here is the dump. Note that I am over-writing the next descriptor but for now I dont care as I am only interested in getting the first packet out then I can resolve that issue later.
    00000051 00000000 890002E0 00000051
    108047E6 00000000 00000100 80011500
    00000000 11112222 33334444 11112222
    0080000E 0014000A 00000000 00800022
    002F0006 1B240000 67000000 00000000

    00000000 00000000 00000100 80011600

    I also gather that for the case of UDP checksum, I actually have to pass in the checksum for the IPV4 psuedo header as I tried it by putting zero for the UDP checksum only and I was out by the pseudo header in the checksum computation. I then manually computed the IPv4 pseudo header checksum and initialized it to that and then the received packet in my capture had its UDP checksum match.

    [Eric] Yes, you do need to provide IP psudo checksum, there is no protocol awareness in the UDP/IP checksum. The Checksum IP will just sum up all 16-bit words within the checksum range.

    [Aamir] thanks

    Another thing I discovered is that if I put a breakpoint in the Send function at the pa_formatTxRoute() and manually step through the code till the point where the packet is actually pushed out to the EMAC it works but if I put a breakpoint at the point where it is pushed out instead then it does not work and I see no packets going out of the DSP- That seems very strange - Any explanation?

    [Eric] You may have a cache or buffering problem. Can you send me the psInfo dump prior to packet pushing in both cases.

    [Aamir] You are correct. It seems like for the case when it does not work, my descriptor data is incorrect from what I am expecting. The length of the buffer is wrong (not 0x51) as well as the pointer to where the buffer is is not 0x108047e6. This is the descriptor when it works. However I have found that it works some of the times even when the first stop is in the spot before pushing the data out. Same also when I remove all breakpoints and just send a packet to the DSP, it replies back ok most of the times.

    00000051 00000000 890002E0 00000051
    108047E6 00000000 00000100 80011500
    00000000 11112222 33334444 11112222
    0080000E 0014000A 00000000 00800022
    002F0006 1B240000 67000000 00000000
    00000000 00000000 00000100 80011600

    Note that I have code that triggers an outgoing packet whenever an incoming packet is received by the host.

    I thought I could use my intended descriptor and the one next to it as one descriptor but that is a little complicated (this is tip2 in the user guide). Do you suggest that option is even workable?

    [Eric]  No, I do not think that it is a good idea since you are going to overwrite the adjacent CPPI descriptor and how do you plan to recover them at the end.

    [Aamir] Well I too do not want to do that but I have tried to put the PS data prior to the packet data in the buffer pointed to by the descriptor but I have not got that to work for some reason as the PS data is still being placed in the descriptor. Can I use the Cppi_setPSData function to attach the PS data to the start of the SOP or does this function only work for placing it in the descriptor?

    Best regards,

    Eric

  • Eric,

    Just an update on the last point above. I was able to get my code to work with PS data prior to the SOP by using memcpy to put the 36 byte command prior to the SOP in the buffer and not making a call to the Cppi_setPSData function. I also had to call Cppi_setPSLen for the descriptor with the length set to 36 bytes and then I got the packets out.

    Maybe the version of code I have does not alow the Cppi_setPSData function to work when not having PS data in the desc.

    Thanks, Aamir

  • Hi, Aamir:

    Thanks for the updates! Actually, it is clearly stated that the MACRO Cppi_setPSData() does not support PS Data in SOP at its comments. All the data in CPPI descriptors is in the form of 32-bit words and the hardware will perform Endianness conversion, however, the data buffer is in 8-bit byte mode. Therefore, it is up to the application to perform Endianness conversion if necessary. What you did is correct if you want to store PS data in the SOP buffer.

    Please note that you can create multiple CPPI regions with different size of descriptors and push those descriptors to different queues. You can then use different size of descriptors based on your use case.

    Best regards,

    Eric

  • Eric,

    Thanks for the info. Sorry I actually did not look at the Cppi_setPSData macro code otherwise it would have made sense.

    Thanks for the advice re: multiple CPPI regions.

    Aamir

  • Eric,

    I had one other minor question. When I make a call to the Pa_formatTxRoute with the checksum arguments as NULL, the command returned is 12 bytes long and deals only with the next route I believe. However the struct pasahoNextRoute_t is actually 16 bytes long. Can you explain the discrepancy?

    Thanks, Aamir

  • Aamir:

    The maxmium szie of nextRoute command is 16-byte. It could be either 12-byte or 16-byte. It will be 16-byte only if you want to specify packet type for SRIO, emac port control for CPSW.

    Best regards,

    Eric

     

     

  • Eric,

    Hi! Pinging you. Do you have any insight on the discussion on the attached thread.

    http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/197556/856527.aspx#856527

    Thanks, Aamir

  • Hi, Aamir:

    Unfortunately, it is not my area of expertise. I am sure that Javier will answer your questions in a timely mannar.

    Best regards,

    Eric