SRIO communication between two c6678 dsps through switch

Deepankar Debnath

We have four c6678 DSPs(with SRIO port 0 of each dsp only) connected to a switch(cps1616) on a custom board. Our aim is to establish communication between any two dsp, say DSP0 and DSP1 through the switch. Our desired configuration is 1x and 2.5Gbps speed. We tried running loopbackDioIsr project on DSPs first in loopback mode and they were successful . We made the necessary changes to run loopbackDioIsr project in normal mode by changing mode from loopback to normal and swapped the source and destination ID in second dsp, DSP1. We made routing table in switch through switch registers. We followed this thread: We also made sure that before any transfer occur, respective ports in switch are in OK state. The problem is that, in srio switch as soon as transfer from DSPs starts, the port shows output error and input error. We tried to run the code snippet(maintenanceReadReg) given in above thread by Brandy but it returned error that "ISR didn't happen within set time". We contacted IDT too for this error and they are suggesting us to do routing through maintenance packets but I have little idea about how to do it.

I have the following questions:

1. Is there any register which stores number of packets sent through DSP? So that we
can confirm that some packets have been sent out of DSP.

2. How can we configure switch through maintenance packets?

Some of the register values after we run above program in normal mode are:
rio_sp_err_stat= 0x00020002
rio_sp_ctl = 0x00600001
rio_err_det = 0x01000000
rio_sp_err_det = 0x00000001
rio_sp_lm_resp = 0x80000005
rio_comp_tag = 0x00000000
Please let us know if you need to any other register values.

over 10 years ago

0 Raja over 10 years ago

TI__Guru* 81335 points

Welcome to the TI E2E forum. I hope you will find many good answers here and in the TI.com documents and in the TI Wiki Pages (for processor issues). Be sure to search those for helpful information and to browse for the questions others may have asked on similar topics (e2e.ti.com).

We will get back to you on the above query shortly. Thank you for your patience.

0 Ganapathi Dhandapani95 over 10 years ago

TI__Mastermind 28085 points

Hi,

Please take a look at MCSDK SRIO Throughput test code. It support external SRIO switch.

MCSDK Path: C:\ti\pdk_C6678_1_1_2_6\packages\ti\drv\exampleProjects\SRIO_TputBenchmarkingTestProject

Refer section "5.4 Single EVM looped back externally using an external SRIO switch" and "9.3 Setting up C-S-C connection mode (core to core, with a SRIO switch)" on SRIO_Benchmarking_Example_Code_Guide document.

Doc Path: \ti\pdk_C6678_1_1_2_6\packages\ti\drv\srio\test\tput_benchmarking\docs\SRIO_Benchmarking_Example_Code_Guide.

Thanks,

0 Deepankar Debnath over 10 years ago in reply to Ganapathi Dhandapani95

Prodigy 110 points

Hi Ganapathi,

Thanks for your reply. I tried to run SRIO_TputBenchmarkingTestProject on DSP in loopback mode. It seems there is some problem with transmission of packets. I have attached screenshot of output here. Sorry for the formatting.

0 Ganapathi Dhandapani95 over 10 years ago in reply to Deepankar Debnath

TI__Mastermind 28085 points

Hi,

Have you modified any function on this example?

Have you referred SRIO_Benchmarking_Example_Code_Guide for build and run the SRIO example as loopback mode?

I have tested the SRIO example project (SRIO_TputBenchmarkingTestProject) on EVM C6678 in loopback mode. It is working fine. Please find the attached test log.

Fullscreen SRIO_Tput.txt Download

[C66xx_0] ********************************
[C66xx_0] *********** CONSUMER ***********
[C66xx_0] ********************************
[C66xx_0] WARNING: Please ensure that the CONSUMER is executing before running the PRODUCER!!
[C66xx_0] Debug: Waiting for module reset...
[C66xx_0] Debug: Waiting for module local reset...
[C66xx_0] Debug: Waiting for SRIO ports to be operational...  
[C66xx_0] Debug: SRIO port 0 is operational.
[C66xx_0] Debug:   Lanes status shows lanes formed as one 4x port
[C66xx_0] Debug: AppConfig Tx Queue: 0x2a0 Flow Id: 0
[C66xx_0] Debug: SRIO Driver Instance 0x@00861840 has been created
[C66xx_0] Debug: Running test in polled mode.
[C66xx_0] Debug: SRIO Driver handle 0x861840.
[C66xx_0] 
[C66xx_0] 
[C66xx_1] ********************************
[C66xx_1] *********** PRODUCER ***********
[C66xx_1] ********************************
[C66xx_1] WARNING: Please ensure that the CONSUMER is executing before running the PRODUCER!!
[C66xx_1] Debug(Core 1): Waiting for SRIO to be initialized.
[C66xx_1] Debug: AppConfig Tx Queue: 0x2a1 Flow Id: 1
[C66xx_1] Debug: SRIO Driver Instance 0x@00861750 has been created
[C66xx_1] Debug: Running test in polled mode.
[C66xx_1] Debug: SRIO Driver handle 0x861750.
[C66xx_1] 
[C66xx_1] 
[C66xx_1] Latency: (DIO_NW, 5.000GBaud, 4X, tab delimited)
[C66xx_1] Core	Lanes	Speed	Conn	MsgType	PktSize	NumPkts	MnLCycs	AgLCycs	MxLCycs
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	4	100	683	684	736
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	8	100	683	683	695
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	100	698	698	710
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	32	100	719	728	749
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	64	100	791	791	802
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	128	100	899	911	917
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	256	100	1150	1151	1151
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	512	100	1311	1313	1313
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	1024	100	1601	1619	1636
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	2048	100	2213	2222	2231
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	4096	100	3437	3437	3453
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	8192	100	5867	5867	5883
[C66xx_1] 
[C66xx_0] Throughput: (RX side, DIO_NW, 5.000GBaud, 4X, tab delimited)
[C66xx_0] Core	Lanes	Speed	Conn	MsgType	OHBytes	PktSize	Pacing	Thruput	PktsSec.	NumPkts 	PktLoss	AgPCycs	AgLCycs	AgICycs	AgOCycs	Seconds
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	4	0	85.33	2666666.75	12600000	No	375	21	354	0	4.73
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	8	0	170.67	2666666.75	12600000	No	375	21	354	0	4.73
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	16	0	341.33	2666666.75	12600000	No	375	21	354	0	4.73
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	32	0	682.67	2666666.75	12600000	No	375	21	354	0	4.73
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	64	0	1365.33	2666666.75	12600000	No	375	21	354	0	4.73
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	128	0	2730.67	2666666.75	12600000	No	375	21	354	0	4.73
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	256	0	5461.33	2666666.75	12600000	No	375	21	354	0	4.73
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	512	0	10922.67	2666666.75	12600000	No	375	21	354	0	4.73
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	1024	0	13191.63	1610306.00	7800000	No	621	21	600	0	4.84
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	2048	0	13223.57	807102.50	4000000	No	1239	21	1218	0	4.96
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	4096	0	13277.15	405186.38	2000000	No	2468	21	2447	0	4.94
[C66xx_0] 0	4	5.000	C-I-C	DIO_NW	16	8192	0	13323.03	203293.36	1200000	No	4919	21	4898	0	5.90
[C66xx_0] 
[C66xx_1] Throughput: (TX side, DIO_NW, 5.000GBaud, 4X, tab delimited)
[C66xx_1] Core	Lanes	Speed	Conn	MsgType	OHBytes	PktSize	Pacing	Thruput	PktsSec.	NumPkts 	PktLoss	AgPCycs	AgLCycs	AgICycs	AgOCycs	Seconds
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	4	0	85.33	2666666.75	12600000	No	375	319	41	15	4.73
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	8	0	170.67	2666666.75	12600000	No	375	319	41	15	4.73
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	16	0	341.33	2666666.75	12600000	No	375	319	41	15	4.73
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	32	0	682.67	2666666.75	12600000	No	375	319	41	15	4.73
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	64	0	1365.33	2666666.75	12600000	No	375	319	41	15	4.73
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	128	0	2730.67	2666666.75	12600000	No	375	319	41	15	4.73
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	256	0	5461.33	2666666.75	12600000	No	375	319	41	15	4.73
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	512	0	10922.67	2666666.75	12600000	No	375	319	41	15	4.73
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	1024	215	13191.63	1610306.00	7800000	No	621	317	289	15	4.84
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	2048	831	13223.57	807102.50	4000000	No	1239	319	905	15	4.96
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	4096	2061	13271.77	405022.28	2000000	No	2469	317	2137	15	4.94
[C66xx_1] 1	4	5.000	C-I-C	DIO_NW	16	8192	4513	13320.33	203252.03	1200000	No	4920	318	4587	15	5.90
[C66xx_1] 
[C66xx_1] Latency: (DIO_NR, 5.000GBaud, 4X, tab delimited)
[C66xx_1] Core	Lanes	Speed	Conn	MsgType	PktSize	NumPkts	MnLCycs	AgLCycs	MxLCycs
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	4	100	893	894	983
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	8	100	893	895	983
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	16	100	893	962	997
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	32	100	893	970	1032
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	64	100	893	1034	1068
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	128	100	893	1185	1214
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	256	100	893	1414	1455
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	512	100	893	1545	1632
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	1024	100	893	1866	1879
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	2048	100	893	2470	2505
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	4096	100	893	3758	3793
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	8192	100	893	6597	6599
[C66xx_1] 
[C66xx_1] Throughput: (TX side, DIO_NR, 5.000GBaud, 4X, tab delimited)
[C66xx_1] Core	Lanes	Speed	Conn	MsgType	OHBytes	PktSize	Pacing	Thruput	PktsSec.	NumPkts	PktLoss	AgPCycs	AgLCycs	AgICycs	AgOCycs	Seconds
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	4	0	47.20	1474926.25	7200000	No	678	324	339	15	4.88
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	8	0	94.40	1474926.25	7200000	No	678	324	339	15	4.88
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	16	0	188.51	1472754.00	7200000	No	679	324	340	15	4.89
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	32	0	353.59	1381215.50	6800000	No	724	327	382	15	4.93
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	64	0	656.41	1282051.25	6400000	No	780	324	441	15	5.00
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	128	0	1137.78	1111111.13	5600000	No	900	324	561	15	5.04
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	256	0	1771.63	865051.88	4200000	No	1156	325	816	15	4.86
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	512	0	3133.89	765110.94	3800000	No	1307	324	968	15	4.97
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	1024	0	5072.45	619195.06	3200000	No	1615	328	1272	15	5.17
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	2048	0	7383.51	450653.44	2400000	No	2219	323	1881	15	5.33
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	4096	0	9389.11	286532.94	1600000	No	3490	324	3151	15	5.58
[C66xx_1] 1	4	5.000	C-I-C	DIO_NR	28	8192	0	10400.89	158704.97	800000	No	6301	322	5964	15	5.04
[C66xx_1] 
[C66xx_1] Latency: (Type-11, 5.000GBaud, 4X, tab delimited)
[C66xx_1] Core	Lanes	Speed	Conn	MsgType	PktSize	NumPkts	MnLCycs	AgLCycs	MxLCycs
[C66xx_1] 1	4	5.000	C-I-C	Type-11	16	100	1424	1500	1612
[C66xx_1] 1	4	5.000	C-I-C	Type-11	32	100	1458	1549	1581
[C66xx_1] 1	4	5.000	C-I-C	Type-11	64	100	1607	1632	1662
[C66xx_1] 1	4	5.000	C-I-C	Type-11	128	100	1731	1801	1836
[C66xx_1] 1	4	5.000	C-I-C	Type-11	256	100	2055	2099	2105
[C66xx_1] 1	4	5.000	C-I-C	Type-11	512	100	2112	2165	2226
[C66xx_1] 1	4	5.000	C-I-C	Type-11	1024	100	2435	2499	2537
[C66xx_1] 1	4	5.000	C-I-C	Type-11	2048	100	3191	3207	3230
[C66xx_1] 1	4	5.000	C-I-C	Type-11	4096	100	4512	4530	4646
[C66xx_1] 
[C66xx_0] Throughput: (RX side, Type-11, 5.000GBaud, 4X, tab delimited)
[C66xx_0] Core	Lanes	Speed	Conn	MsgType	OHBytes	PktSize	Pacing	Thruput	PktsSec.	NumPkts	PktLoss	AgPCycs	AgLCycs	AgICycs	AgOCycs	Seconds
[C66xx_0] 0	4	5.000	C-I-C	Type-11	24	16	0	175.10	1367989.00	6800000	No	731	589	32	110	4.98
[C66xx_0] 0	4	5.000	C-I-C	Type-11	24	32	0	346.88	1355013.50	6800000	No	738	596	32	110	5.02
[C66xx_0] 0	4	5.000	C-I-C	Type-11	24	64	0	696.60	1360544.25	6800000	No	735	593	32	110	5.00
[C66xx_0] 0	4	5.000	C-I-C	Type-11	24	128	0	1398.91	1366120.25	6800000	No	732	589	33	110	4.98
[C66xx_0] 0	4	5.000	C-I-C	Type-11	24	256	0	2801.64	1367989.00	6800000	No	731	589	32	110	4.98
[C66xx_0] 0	4	5.000	C-I-C	Type-11	24	512	0	5565.22	1358695.63	6800000	No	736	591	33	112	5.00
[C66xx_0] 0	4	5.000	C-I-C	Type-11	24	1024	0	11055.33	1349527.63	6800000	No	741	592	39	110	5.04
[C66xx_0] 0	4	5.000	C-I-C	Type-11	24	2048	0	11636.36	710227.25	3600000	No	1408	598	700	110	5.07
[C66xx_0] 0	4	5.000	C-I-C	Type-11	24	4096	0	11640.50	355239.78	1800000	No	2815	594	2110	111	5.07
[C66xx_0] 
[C66xx_1] Throughput: (TX side, Type-11, 5.000GBaud, 4X, tab delimited)
[C66xx_1] Core	Lanes	Speed	Conn	MsgType	OHBytes	PktSize	Pacing	Thruput	PktsSec.	NumPkts	PktLoss	AgPCycs	AgLCycs	AgICycs	AgOCycs	Seconds
[C66xx_1] 1	4	5.000	C-I-C	Type-11	24	16	214	175.10	1367989.00	6800000	No	731	140	574	17	4.98
[C66xx_1] 1	4	5.000	C-I-C	Type-11	24	32	214	346.88	1355013.50	6800000	No	738	140	581	17	5.02
[C66xx_1] 1	4	5.000	C-I-C	Type-11	24	64	230	696.60	1360544.25	6800000	No	735	140	578	17	5.00
[C66xx_1] 1	4	5.000	C-I-C	Type-11	24	128	215	1398.91	1366120.25	6800000	No	732	140	575	17	4.98
[C66xx_1] 1	4	5.000	C-I-C	Type-11	24	256	216	2801.64	1367989.00	6800000	No	731	140	574	17	4.98
[C66xx_1] 1	4	5.000	C-I-C	Type-11	24	512	228	5565.22	1358695.63	6800000	No	736	140	579	17	5.00
[C66xx_1] 1	4	5.000	C-I-C	Type-11	24	1024	233	11055.33	1349527.63	6800000	No	741	144	580	17	5.04
[C66xx_1] 1	4	5.000	C-I-C	Type-11	24	2048	0	11644.63	710732.06	3600000	No	1407	145	1245	17	5.07
[C66xx_1] 1	4	5.000	C-I-C	Type-11	24	4096	0	11640.50	355239.78	1800000	No	2815	144	2654	17	5.07
[C66xx_1]

Thanks,

0 Deepankar Debnath over 10 years ago in reply to Ganapathi Dhandapani95

Prodigy 110 points

Hi,
SRIO_TputBenchmarkingTestProject is working fine in loopback mode. Actually in our board we are using 156.25MHz as ref clock but in srio_device_tput.c file ref clk is directly assigned 312.5MHz. But it seems that above project is working properly in loopback mode without making this change. When we tried to connect our DSP to srio switch(cps1616) then without changing this value, we were not able to get "port ok" on switch(We were getting input error on port). Now when we changed it(ref clk value in srio_device_tput.c), errors also mitigated.
We are trying to use C-S-C configuration, but as soon as DSP is starting to send packets, the switch port is showing input error. We ensured that routing table is as per our requirement. From counter registers in switch we found that it received only 1 packet that too it dropped. The same thing happened when we tried to run diolsrloopback project in normal mode.

Processors

Processors forum

SRIO communication between two c6678 dsps through switch