This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

PA Tx Queues and sending packets out

Hi TI Folks,

On our current TI c64x based product we make use of the EDMA engine to send packets of the DSP. The packets for multiple channels are stored in buffers in external memory and then each of these buffers is sent out one at a time with a DMA wait before the next EDMA send happens if the previous EDMA has not completed sending the packet out from the DSP to an FPGA. These buffers are setup at initialization and are used by each channel for the duration of the time the DSP is running. Going forward to our new product and having these packets sent out through the TX queues using the PKTDMA to the Ethernet switch etc, I see one design issue. I would like to have this buffers fixed per Channel for the duration and so I could grab at initialization a TX buffer descriptors (by popping them of the TX Free Queue) and buffers to hold the Ethernet packet have those pushed onto the TX queue and have them sent to another core or out through the DSP switch. However,  when the sending is complete then I do not want them popped back to the TXFreeQueue as I want the buffers to physically be at the same location for subsequnet use by that channel. Can I do that? This avoids transferring these packet buffers from DDR3 for each channel making use of the EMDA engine to a set of TX descriptors/TX buffers that can be used for sending packets out the chip and then pushing the descriptors to the TX queue for sending via the PKTDMA engine.

Any thoughts.

Thanks, Aamir

  • Aamir,

    The BIOS forum does not seem well suited for this query.  I have move this thread over to the multicore forum in hopes that it will get a faster response there.

    Dave

  • David,

    Thanks. I had actually reposted it to the c66x multicore forum as I had inadvertently posted it on the BIOS forum thinking I was on the c66x one.

  • Aamir,

    Not sure I completely follow you, but what I think you are asking is whether you can setup dedicated buffers within DDR3 that are specific to a given TX ethernet channel.  The answer is yes.  For example let's say you wanted to use 4 TX ethernet channels (i.e. 4 TX queue IDs that interface to the PKTDMA associated with the NetCP).  To do this you would want to do the following:

    - initially dedicate 4 regions of buffers, 1 region per TX channel.  Regions are sized based on how many total buffers you want for each TX channel

    - initially dedicate 4 TXFreeQueues, each free queue corresponds to one of the 4 buffer regions.  When you create (push) each descriptor onto the TXFreeQueue, I suggest hardcoding the buffer to the descriptor at that point, and also setting the return QID in the descriptor to the same TXFreeQueue it came from originally.

    - When a master (for example CPU) wants to send a message on a given TX channel, it must pop a free descriptor from the corresponding TXFreeQueue for that channel.  It will then read the buffer location and use that buffer for the message payload.  When the buffer is written and descriptor completed, it pushes the descriptor to the TX channel and is sent.  After being sent the PKTDMA hardware returns the descriptor to the TXFreeQueue associated with the return QID.  At this point it can be reused.

    By using this method, i.e. a free descriptor queue per buffer region (essentiall per TX channel), the only software requirement other than the original setup of descriptors and queues, is populating the buffer associated with the descriptor as it is needed.  If you didn't want to do this, you could simply have one TXFree queue for all the TX channels and populate the buffer location in the descriptor as you need them.  Essentially, there is no copy of your payload from original location to the predefined buffer location for this method, but software will have to do some extra work polling to figure out when the buffer/descriptor is freed up after sending.

    Regards,

    Travis

  •  Travis,

    Thanks for your note. I have a duplicate posting on the e2e multicore forum http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/224502/794827.aspx#794827. Please do have a look. I would appreciate it. We can then close this one.

    It would appear that I can have a Tx FDQ for each of the 128 channels though that is not an option I prefer if I do not have to do so.

    I want to know if I can reuse a descriptor for subsequent time periods where new data is being placed in the buffers pointed to by the descriptor and if I can not have the buffer go back onto the Tx FDQ (as specified by the return Queue) as some other entity could then use it when popping descriptors and I do not want this channel to pop this descriptor again but keep reusing it again and again and pushing it onto the Tx Q.

    Thanks, Aamir

  • Aamir Husain said:
    Please do have a look. I would appreciate it. We can then close this one.

    I'll take a look shortly.

    Aamir Husain said:

    It would appear that I can have a Tx FDQ for each of the 128 channels though that is not an option I prefer if I do not have to do so.

    I want to know if I can reuse a descriptor for subsequent time periods where new data is being placed in the buffers pointed to by the descriptor and if I can not have the buffer go back onto the Tx FDQ (as specified by the return Queue) as some other entity could then use it when popping descriptors and I do not want this channel to pop this descriptor again but keep reusing it again and again and pushing it onto the Tx Q.

    The only additionally way I can think of is to have one TX FDQ which is available for all TX channels.  Once you initially pop a descriptor from this TX FDQ for use on a given channel, you edit the descriptor's return queue before pushing it to the TX channel queue.  The return queue you specify would need to be owned by a given master (dsp core) that would then edit the descriptor and push it back on to the TX channel queue for the next transmit operation.  Maybe nothing is editted at all in the descriptor, i.e. the buffer is reused and everything remains the same, it is simply pushed to the TX channel queue.  Or, maybe the descriptor is pointed to a new buffer altogether before being pushed.  Either way, as long as the return queue is not changed, that dsp core will own that descriptor since it will be the only entity that uses it.  Once the dsp core is ready to give up the descriptor and buffer, it edits the return queue to once again point to the original TX FDQ.

    There is no way for the TX channel queue to reuse the same descriptor over and over without intervention from some other entity pushing the descriptor on that queue.  You can not specify the descriptor's return queue as the TX channel queue itself, as this would be a non-ending loop.

    Regards,

    Travis

  • Travis,

    Travis,

    I am not sure I understand your possible additional way enough but here is my stab at it. Are you suggest that some other entity on the same core receive the buffer descriptor by popping it from its queue when the tx completes, modify it if necessary and then push it to a given channel (I have 128 of these for RTP data). The given channel then ends up popping the same descriptor as it used the last time and so also the same buffer pointed to by the descriptor. Then when it is ready to transmit some new data out through the Ethernet it pushes to the Tx queue. Or are you actually suggesting that another core (entity) handle the external interventation that is needed. Seems like this is really too complicated and I might be better served having static buffers in external memory for each of my RTP channels and then when I want to transmit one of these, i pop a descriptor, copy the data from the RTP channel buffer to the buffer pointed by the descriptors and then push the descriptor to the TX queue. I probably only need a few of these buffers as the time it takes to send them out over the Ethernet is probably faster than the time it takes for me to move to the next RTP channel processing and request for another descriptor and buffer. The downside is the added cycle count for ensuring that the transfer is complete if using EDMA or something similar.

    Thanks, Aamir

  • Aamir Husain said:
    Are you suggest that some other entity on the same core receive the buffer descriptor by popping it from its queue when the tx completes, modify it if necessary and then push it to a given channel (I have 128 of these for RTP data). The given channel then ends up popping the same descriptor as it used the last time and so also the same buffer pointed to by the descriptor. Then when it is ready to transmit some new data out through the Ethernet it pushes to the Tx queue

    I think we are saying the same thing here.  Bottom line is that after the initial transmission, the CPDMA will return the descriptor to the queue pointed at by the descriptor return queue field.  Something will have to resubmit the descriptor to the TX queue for the next transmission.  Before resubmitting it, it can either leave the descriptor as is (pointing to the same previous buffer location), or it can be modified to point to a new one.  Just depends how you want to manage that on the SW side.

    Regards,

    Travis