• State Resolved
  • Locked Locked
  • Replies 4 replies
  • Subscribers 102 subscribers
  • Views 759 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

SRIO communication between cores on multiple c6678 chips

Aamir Husain
Expert 2715 points

Hi,

We intend to have our application run on each core of the c6678 chip on a board with multiple chips. One possible future hardware configuration is a farm with 4 chips with each chip having point-to-point SRIo connectivity to their neighbouring chips but the non-neighbouring chip can be accessible through the use of SRIO packet forwarding option. The issues I see in implementing sRIO communication have to do with specifying the SRIO global IDs.

Each core of each chip will have its own unique SRIO global ID so if any of the 8 cores which are running their own application on chip 1 want to communicate with any of the 8 cores on chip 2 by making use of SRIO. One can specify the port (lane) that they will be using assuming 1x configuration and the unique global SRIO source and dest ID which can be an 8-bit number and they should be able to send packets to a specific core on the other chip. However with the packet forwarding in hardware option available for SRIO, how can i get the dest ID to match one of 8 device IDs of each of the cores on that particular chip. Similarly, when I want to forward to the neighbour chip and its 8 core IDs, how can I go about doing that. If you disable packet forwarding.

Also for the case where I do not support packet forwarding, I want to make sure that if teh dest chip receives packet using any of the 8 SRIO global IDs of the core on the destination chip, that the packet will be accepted by the chip and get routed (stored in descriptors and buffers for each core).

Thanks, Aamir

  • 0
    TI__Mastermind 23525 points

    Aamir,

    What you are trying to do IS possible, but you need to first set some expectations.  This type of system is really for lower bandwidth systems since you are sharing link resources where forwarded traffic is contending for the same resources as direct traffic.  Additionally, packet types without responses would help reduce the amount of traffic on the shared links.   I don't have numbers to share here, it will depend on many factors, i.e. number of devices/hops, traffic types.  I'm just trying to set expectations, if you want performance, use a switch. 

    To answer your questions.  First, you must setup the RX DESTID checking mode.  There are three control bits that need to be set, See Table 2-2 in http://www.ti.com/lit/ug/sprugw1b/sprugw1b.pdf.  You will most likely want mode C for forwardig.  Next you will want to setup the RIO_BASE_ID and Base Routing Registers (RIO_PLM_SPn_BRR_n_PATTERN_MATCH & RIO_PLM_SPn_BRR_n_CTL).  Please see this post for more details: http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/261751.aspx.  And of course for the forwarding portion you must program the forwarding register RIO_PF_8b_Cntlx/RIO_PF_16b_Cntlx registers.  Please review section 2.3.13 in the user's guide above for understanding how the matching and handling of incoming packets work.

    Once the setup is correct, all you have to do is make sure the DestID in the packet is the same as the one used in RIO_BASE_ID or Base routing registers to determine acceptance and fowarding.

    Regards,

    Travis

  • 0 in reply to tscheck
    Expert 2715 points

    Travis,

    Thanks for your answers. My question on the packet forwarding was a little forward thinking for a future product so it is not something we need to address anytime soon.

    Our current product does have an SRIO switch with 8-12 c6678's connected to it. We will have the routing table on the switch programmed to forward multiple SRIO dest IDs to a particular port on the switch for DSP to DSP communication via the switch. I had a few more questions in regards to table 2-2 and what setting I needed to use from there since I do not want to do packet forwarding, it would rule out mode C. I would think it would need to be set to E or F and so all packets to SRIo dest ID1-8 wil be sent to chip 1 and packets to SRIo dest ID 9-16 will be sent to chip 2 and so on and within the SRIO module all packets irrespective of SRIo destID get sent up to the logical layer. Can you summarise the difference between these modes. I do not quite get the explanations given. Sorry, Iam not much familiar with SRIO and the SRIO user guide I find is a really difficult read.

     I then need to have these packets get directed to the PA. I would like to confirm that I can get these packets all forwarded to the PA?  The PA will then make use of the SRIO dest ID in the PDSP0 i.e. LUT1 stage to have packets go to the appropriate core so that destID=1 will go to core 0 and destID-2 will go to core 1 and so on. Looking through the message passing section i.e. 2.3.3 of the srio user guide for keystone, it shows the ability to get packets sent to the PA. I have specified in the PA rx flow to have each core use different queues i.e. 900 for core0, 901 for core 1 etc. Can I do the same thing in setting up the Rx flows for the SRIO pktdma and so I do not need to have the packets go to the PA from the SRIO unit and instead the SRIO pktdma puts them into the right rx queues for each core and they end up in DDR3 for each core to process independently? It would appear so from the example SRIOmulticoreExample in the srio subdirectory under the pdk that comes with my mcsdk so I am not sure of why the PA has this SRIO capability as the online training etc on the PA did not mention anything to do with SRIO. The use of SRIO is only mentioned in the PA LLD api document so it appears that this capability was added to the PA later.

    Does message passing mode always have message responses from the destination entity? I want to be able to send data between c6678 devices using Ethernet and if bandwidth is a concern use SRIO instead so I am sending really large data files such as a HD 720p YUV video picture. I do not think I need to have message response so should I be using message type 9?

    Is their any example code from TI that illustrates packets being sent using SRIO and then being received back and forwarded onto the PA and then back to the corePacs? I do not think test 9 in the PA unit test covers that?

    Thanks, Aamir

  • 0 in reply to Aamir Husain
    TI__Mastermind 23525 points

    Aamir Husain said:
    Our current product does have an SRIO switch with 8-12 c6678's connected to it. We will have the routing table on the switch programmed to forward multiple SRIO dest IDs to a particular port on the switch for DSP to DSP communication via the switch. I had a few more questions in regards to table 2-2 and what setting I needed to use from there since I do not want to do packet forwarding, it would rule out mode C. I would think it would need to be set to E or F and so all packets to SRIo dest ID1-8 wil be sent to chip 1 and packets to SRIo dest ID 9-16 will be sent to chip 2 and so on and within the SRIO module all packets irrespective of SRIo destID get sent up to the logical layer. Can you summarise the difference between these modes. I do not quite get the explanations given. Sorry, Iam not much familiar with SRIO and the SRIO user guide I find is a really difficult read.

    I'm guessing you meant table 2-22.  So here is the way to think about these three control bits and the modes they allow:

    mtc_tgt_id_dis - controls whether a type 8 maintenance packet is intercept and responded to by the SRIO physical layer hardware. (0b1)  any DESTID is responded to so no packet forwarding of type 8 packets, or (0b0) only the 16 supported DESTIDs are responded too and any other is forwarded to the logical layer, where further decisions can be made based on log_tgt_id_dis and the packet forwarding routing tables.

    tgt_id_dis - controls whether the SRIO physical layer forwards a non-type 8 data packets to the logical layer or not.  (0b1) all non-type 8 packets are forwarded to the logical layer regardless of DESTID, where further decisions can be made based on log_tgt_id_dis and the packet forwarding routing tables.  (0b0) only the 16 supported DESTIDs are forwarded to the logical layer, so no forwarding.

    log_tgt_id_dis - controls whether the packets that have reached the logical layer are for the local device or not.  Essentially it controls the promiscuous mode.  (0b1) All packets are accepted locally, so no packet forwarding. (0b0) only packets that match the 16 DESTIDs or 8 MULTICAST IDs are for local consumption, the other packets will either be forwarded or destroyed based on the packet forwarding routing tables.

    So for your case mode, if you use Mode F, and just rely on the Switch itself to direct the correct DESTIDs to the DSP, you will be fine.

    Aamir Husain said:
    Can I do the same thing in setting up the Rx flows for the SRIO pktdma and so I do not need to have the packets go to the PA from the SRIO unit and instead the SRIO pktdma puts them into the right rx queues for each core and they end up in DDR3 for each core to process independently? 

    Yes, you do not need the PA at all, the RXU mapping tables allow you to route specific packets to specific destination Queues (and ultimately the core) directly.  Take a look at the mapping registers, which give good flexibility.  PA is used more for ethernet and routing based on UDP port, etc.

    Aamir Husain said:
    I do not think I need to have message response so should I be using message type 9?

    Type 11 have responses, type 9 do not.

    Aamir Husain said:
    Is their any example code from TI that illustrates packets being sent using SRIO and then being received back and forwarded onto the PA and then back to the corePacs? I do not think test 9 in the PA unit test covers that?

    Again, you shouldn't need the PA.  There isn't an example that I know of for this,  I will ask around.

    Regards,

    Travis

  • 0 in reply to tscheck
    Expert 2715 points

    Travis,

    Thanks for the answers. I will try what you suggest and get back if I have any implementation issues. I was hoping to understand the purpose of the SRIO capability within the PA? It appears more flexible. I was hoping Eric could shed some more light to its capability.

    Thanks, Aamir