C6678 segmentation contexts

Hi,

I already read this user guide but  I still don't understand the way the segments are discovered by the SRIO peripheral. 

Each type 9 message is divided by the HW to several segments (depends on message size and system MTU size): The 1st segment header include the following fields:  SrcID + DestID + COS + StreamID, All other segments of that message include only the following fields: SrcID+DestID+COS.

A segmentation context is opened when the 1st message segment arrived and all other segments use this context, but it is not clear from the User guide which fields of the header are used to identify that contexts. 

In "RapidIO: The Embedded System Interconnect"  (By Sam Fuller) book in page 204 it seems that the context depends on DestID+COS+StreamID and according to the C6678 datasheet, the device supports up to 16 context 

BUT when I tried it on my system as described in my original question (Each transmitter DSP send a message with different COS and different Stream ID but with the same destID and there are only 3 transmitter --> only 3 context should be used) I still got the context errors I described. 

My receiver DSP is configured to use the same reception flow for all the messages (COS mask=0; Stream mask=0; Src_Prom=1;dest_prom=1)

What do I miss in my understanding of the reassembly mechanism? What header fields are used to identify the segmentation context?

Thanks,

Hi, Thanks for your reply.

How can i know how many segmentation context are enable? How do I enable all segmentation context? 

I look for it in TI examples and didn't see any initialization that deal with the segmentation context initialization? 

Thanks

Hi,

Thanks for the reply.

I already checked all those issues.

1. Each producer has its own SRCID - I verified that by looking at the PS info of the Rx descriptors at the Rx ISR (for the receive messages), in addition each producer does not aware about the other producers so there is no chance that the same SRCID is used by more than one producer

2. I also test the case that each producer send the its messages with different COS and different STREAMID (I enabled the COS_EN bit you mentioned, and set the flow configuration to COS_MASK=0, STRM_MASK=0) but the results were the same.

3. My system architecture does not allow different DESTID for each consumer and I also can't use different priorities for each producer - so this kind of solution is not good for me.

Regards,

Tzvi

Tzvi E said:

1. Each producer has its own SRCID - I verified that by looking at the PS info of the Rx descriptors at the Rx ISR (for the receive messages), in addition each producer does not aware about the other producers so there is no chance that the same SRCID is used by more than one producer

Just for clarification, you verified that each producer can send a multi-segment message to the receiving device when the other producers are not sending anything?  Essentially is this only a problem when there is overlap of producers trying to send or only when greater than 256B?  If each producer can successfully send a multi-segment  message to the receiving device, we know the mapping registers are setup right.  Segmentation contexts are handled separately from the mapping registers.  For example, you have the SRC and DEST promiscous bits set in the mapping registers, so that it will accept anything, but are you using 8b or 16b IDs (TT field in the PS TX descr)?  Which bits are you using in PS TX desc for SRCID/DESTID if 8b?  This may affect the RX segmentation contexts depending on how you have the IDs setup.

Tzvi E said:

2. I also test the case that each producer send the its messages with different COS and different STREAMID (I enabled the COS_EN bit you mentioned, and set the flow configuration to COS_MASK=0, STRM_MASK=0) but the results were the same.

Ok, so this is telling me the COS doesn't affect the segmentation context.  If this is a segmentation context issue, it can only be SRCID, DESTID, PRIORITY overlap then.

Tzvi E said:

3. My system architecture does not allow different DESTID for each consumer and I also can't use different priorities for each producer - so this kind of solution is not good for me.

Can you even test this configuration at all for debug purposes? 

Hi,

We uses the 8b deviceID. 

What are the correct bits for the PS info of the Tx & Rx descriptors when using 8 bit deviceID? (The UG page 2.3.3 describes only the 16 bits case)

Travis, 

I checked the issues you mentioned:

1. I have no problem sending message from single or two producers to the consumer. The problem occur when 3 producers send the message to the consumer. In addition,  there is no problem if even 4 producers send message less than 256 Bytes, the problem occur only for messages with more than 256 bytes.

2. The PS info in the Tx descriptor is configured properly. (note that the CSL_SRIO_SetType9PSInfo() configured a field called cosMask but this field is not present in the SRIO document - what is the correct form?)

2. I tried to change the priority of the producers so each producer's Tx channel is configured to use different priority - The problem still occurred (but less often)

3. Bit 25 of ERR_DET (message request timeout) is clear, the only bits being set when i lost a packet(s) are bit 9 and bit 10

Can you simulate my producers-consumer setup at your labs and see if the problem occur at your system too?

Regards,

1. Ok, glad to hear the single segment works from multiple producers. Should be no issue there as the incoming packets from different producers will arrive one at a time and can be handled back-to-back. Still not sure why you are seeing issues with multi-segment. Truthfully, there should be no difference between 2 and 3 producer case, as the packet will still be interleaved from the switch if they are really sent simultaneously.

2. good catch, the document needs updating... below is the correct bit description and numbers. Essentially, the TX descriptor COS mask is used to determine if COS is used included in the TX context, similar to the RX COS_EN bit in PER_SET_CNTL1 register.

3. Again, doesn't make sense yet. Different priority would mean different RX segmentation context.

4. Ok, got it.

Unforunately, no we are not setup to with a SRIO switch. I'm in discussions with the SRIO designers to brainstorm.

Regards,
Travis

Is there any chance that you have TX COS masking enabled, but not on the RX side.  If it is included in the TX context but not the RX context then it could cause multiple TX type9 streams with the same srcid and priority to be received and the RX would error them. It might be possible that only with multiple producers does the interleaving occur from a single producer.   This would depend on how many TX queues are you using.  How many are you using on each producer? 

Hi,

Each producer uses a single queue to send the messages and the consumer uses also a single queue to recieve the messages so the interleaveing  occur only for several producers and not for a single producer

I will check the cos masking issue in my Tx and Rx queues but how can it be the problem if every DSP (producer or consumer) uses only single queue?

Regards,

Tzvi

If you are using one TX queue per producer, there is no way the Tx SoC could interleave packets.  The switch shouldn't either if the packets are routed through the same ports and at the same priority to the consumer.

I don't suspect the source of the problem is the Tx (otherwise the switch should drop the bad messages - and it doesn't drop any packet)

The switch route all ingress packets from the different producers (each producer is in different ingress port of the switch) to single egress port (the consumer port). The switch counters shows that all receive packets were sent into the egress port correctly,  but the consumer drop packets and raise the context errors. 

The above information lead me to the conclusion that there is a problem in the Rx mechanism and because the error is "segmentation context error" I assume there is an interleave problem.

Why do you think the switch can't interleave the egress packets when each packet has different sourceID and all packets should be sent from the same port at the same time?

Tzvi E said:

Why do you think the switch can't interleave the egress packets when each packet has different sourceID and all packets should be sent from the same port at the same time?

I'm not thinking this is a problem.  It is totally legal.  I was trying to think of ways that the messages from the same DSP producer somehow become interleaved.  If you are using a single TX queue, there will be no interleaving by the producer.  However, I was trying to think of other potential ways the switch might interleave packets from the same producer.  It shouldn't be the case if all the traffic is at the same priority from given producer.  I'm not aware of all the switch features, but I'm wondering if there are any "time-to-live" counters, or reodering that could be getting done when the traffic through the switch becomes high.  Essentially, you have bottleneck when 3 ingress goes to 1 egress port on the switch.  It is possible to have two messages from the same producer in-flight through the switch at the same time.  Thoughts?  Again, the RX DSP should easily be able to handle 3 producers sending 1 message simultaneously.

Can you please verify on the consumer that you are getting the correct PRIORITY in the RX descriptor?  Please tell me what you read in the RX descriptor.

Each time you send a message, are you opening a new driver instance or just socket?  If it is a driver instance, you are possibly using a new TX queue and not even realizing it.  I will send you a modified PDK project that allows you to specify the TX queue in the Srio_setSockOpt call.  This is a update to the driver that is supposed to go out in the next MCSDK release, but has been delayed for a long time.  The example shows how to set priorities and use multiple TX queues correctly.

Regards,

Travis