This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Runtime Errors of IPC SRIO Example Project on C6678 EVM

Hey,

I'm trying to get the IPC SRIO example project working on C6678 EVM.

For hardware set up: I have 2 C6678 evm boards connected through the dual breakout board, both of them in "no-boot" switch mode.

I've tried the SRIO throughput test, which works. So it seems like the hardware settings are good.

But then When I tried the IPC SRIO example, the console log looks like this:

[C66xx_0] Local Core ("CORE0") starting
Local Core ID: 0
Global Core ID: 0

-----------------------Initializing---------------------------
Core 0 : L1D cache size 4. L2 cache size 0.
Core 0 : Memory region 0 inserted
Port 0 is okay
Port 1 is okay
Port 2 is okay
Port 3 is okay
Core 0: SRIO Driver has been initialized
[C66xx_1] Local Core ("CORE1") starting
Local Core ID: 1
Global Core ID: 1
Core 1: Waiting for SRIO to be initialized.
Core 1: SRIO can now be used.
localQueueName=CORE1
remoteQueueName=CORE3
Core 1: tsk0 starting
[C66xx_0] localQueueName=CORE0
remoteQueueName=CORE2
Core 0: tsk0 starting
Global Core 0: Sending packets to an off-chip core.
Global Core 0 attempting to open remote board Queue CORE2
[C66xx_1] Global Core 1: Sending packets to an off-chip core.
Global Core 1 attempting to open remote board Queue CORE3
[C66xx_0] Local Core ("CORE0") starting
Local Core ID: 0
Global Core ID: 2

-----------------------Initializing---------------------------
Core 2 : L1D cache size 4. L2 cache size 0.
Core 2 : Memory region 0 inserted
Port 0 is okay
Port 1 is okay
[C66xx_1] Local Core ("CORE1") starting
Local Core ID: 1
Global Core ID: 3
Core 3: Waiting for SRIO to be initialized.
[C66xx_0] Port 2 is okay
Port 3 is okay
Core 2: SRIO Driver has been initialized
[C66xx_1] Core 3: SRIO can now be used.
localQueueName=CORE3
Core 3: tsk0 starting
[C66xx_0] localQueueName=CORE2
Core 2: tsk0 starting
Global Core 2: Receiving packets from an off-chip core.
[C66xx_1] Global Core 3: Receiving packets from an off-chip core.
[C66xx_0] ti.sdo.ipc.heaps.HeapBufMP: line 622: E_noBlocksLeft: No more blocks left in buffer (handle = 0x846eb8, requested size = 112)
ti.sdo.ipc.nsremote.NameServerMessageQ: line 227: E_outOfMemory: MessageQ_alloc faild from heap: 0
[C66xx_1] ti.sdo.ipc.heaps.HeapBufMP: line 622: E_noBlocksLeft: No more blocks left in buffer (handle = 0x846eb8, requested size = 112)
ti.sdo.ipc.nsremote.NameServerMessageQ: line 227: E_outOfMemory: MessageQ_alloc faild from heap: 0
...

I've tried to use the exact target configuration file and binary files you built for this project from this post:

 But I still get the same error. Any hints where I should go from here?

Thanks,

Shang

  • Hi Shang,

    I am not able to reproduce your issue on my setup. I have tested the same example on my EVM's with multiple times it is working fine. i already share the test log file on your previous thread.

    Have you got the same error on all the time of testing?

    Take a look at below similar threads:
    e2e.ti.com/.../226234
    e2e.ti.com/.../924099

    Thanks,
  • Hi Ganapathi,

    Thanks for your patience!

    To make sure my hardware is working, I ran the through put test several times with different configurations, seems all working. The log is attached.


    (I set the max packet size as 1024 to save time)

    So I guess the hardware is okay, or is there another way to set up the hardware?

    I also ran the IPC SRIO test several times, sometimes I got the InvalidQueueID error. So I'm really confused..




    Thanks,

    Shang

    srio_throughput_log.txt
    [C66xx_0] ********************************
    *********** CONSUMER ***********
    ********************************
    WARNING: Please ensure that the CONSUMER is executing before running the PRODUCER!!
    Debug: Waiting for module reset...
    Debug: Waiting for module local reset...
    Debug: Waiting for SRIO ports to be operational...  
    Debug: SRIO port 0 is NOT operational.
    Debug:   Lanes status shows lanes formed as one 4x port
    Debug: AppConfig Tx Queue: 0x2a0 Flow Id: 0
    Debug: SRIO Driver Instance 0x@00849120 has been created
    Debug: Running test in polled mode.
    Debug: SRIO Driver handle 0x849120.
    
    
    [C66xx_1] ********************************
    *********** PRODUCER ***********
    ********************************
    WARNING: Please ensure that the CONSUMER is executing before running the PRODUCER!!
    Debug: Waiting for module reset...
    Debug: Waiting for module local reset...
    Debug: Waiting for SRIO ports to be operational...  
    Debug: SRIO port 0 is operational.
    Debug:   Lanes status shows lanes formed as one 4x port
    Debug: AppConfig Tx Queue: 0x2a0 Flow Id: 0
    Debug: SRIO Driver Instance 0x@00849030 has been created
    Debug: Running test in polled mode.
    Debug: SRIO Driver handle 0x849030.
    
    
    Latency: (Type-11, 1.250GBaud, 4X, tab delimited)
    Core	Lanes	Speed	Conn	MsgType	PktSize	NumPkts	MnLCycs	AgLCycs	MxLCycs
    1	4	1.250	B-E-B	Type-11	16	100	772	16564490	1656368589
    1	4	1.250	B-E-B	Type-11	32	100	850	850	872
    1	4	1.250	B-E-B	Type-11	64	100	849	850	872
    1	4	1.250	B-E-B	Type-11	128	100	933	933	957
    1	4	1.250	B-E-B	Type-11	256	100	1096	1098	1119
    1	4	1.250	B-E-B	Type-11	512	100	1180	1180	1202
    1	4	1.250	B-E-B	Type-11	1024	100	1510	1510	1532
    
    [C66xx_0] Throughput: (RX side, Type-11, 1.250GBaud, 4X, tab delimited)
    [C66xx_1] Throughput: (TX side, Type-11, 1.250GBaud, 4X, tab delimited)
    [C66xx_0] Core	Lanes	Speed	Conn	MsgType	OHBytes	PktSize	Pacing	Thruput	PktsSec.	NumPkts	PktLoss	AgPCycs	AgLCycs	AgICycs	AgOCycs	Seconds
    0	4	1.250	B-E-B	Type-11	24	16	0	28.26	220750.55	1200000	No	4530	590	3826	114	5.44
    0	4	1.250	B-E-B	Type-11	24	16	0	26.29	205423.17	1200000	No	4868	590	4164	114	5.84
    0	4	1.250	B-E-B	Type-11	24	16	0	25.56	199720.39	1000000	No	5007	590	4303	114	5.01
    0	4	1.250	B-E-B	Type-11	24	16	0	25.56	199720.39	1000000	No	5007	590	4303	114	5.01
    0	4	1.250	B-E-B	Type-11	24	16	0	25.56	199720.39	1000000	No	5007	590	4303	114	5.01
    Error: RX_1210 Unable to get descriptor for sending the control message
    Error: RX_1338 Unable to get descriptor for sending the control message
    RX_1338 Error: CTRL_READY control failed to be sent.
    Error: RX_1152 Unable to get descriptor for sending the control message
    [C66xx_1] Error: Transmit is unable to send packets. Packet count: 1189265
    Debug:   SRIO status: 0x41800080.
    Debug:     RX CPPI Security Violation
    Debug:     Unsolicited Response
    Debug:     Packet Response Time-out
    Debug:     Message error response
    Debug:   Port response timeout: 0xff0fff
    Debug:   SRIO is an Agent or Slave device.
    Debug:   SRIO processing element can issue requests.
    Debug:   SRIO device has not been previously discovered.
    Debug: Various Register Display:
                                         32211100
                                         17395173
                                         ||||||||
                                         22211000
                                         84062840
                                         --------
                       PCR: 0x02900004:0x00000005
              PER_SET_CNTL: 0x02900014:0x01053800
             PER_SET_CNTL1: 0x02900018:0x00000000
         ERR_RST_EVNT_ICSR: 0x029001e0:0x00000000
                 LSUx_Reg6: 0x02900d18:0x00000010
             LSU_STAT_REG0: 0x02900de8:0x00000000
                 SP_RT_CTL: 0x0290b124:0xff0fff00
                SP_GEN_CTL: 0x0290b13c:0x40000000
      Port n Control 2 CSR: 0x0290b154:0x02aa0000
              SPn_ERR_STAT: 0x0290b158:0x00020002
                   SPn_CTL: 0x0290b15c:0xd0600001
               SPx_ERR_DET: 0x0290c040:0x00000001
              SPx_ERR_RATE: 0x0290c068:0x80000000
               LANEn_STAT0: 0x0290e010:0x00007008
               LANEn_STAT1: 0x0290e014:0x00000000
      PLM Port(n)Implement: 0x0291b080:0x00000000
          PLM_SP(n)_Status: 0x0291b090:0x00000000
           EM_PW_PORT_STAT: 0x0291b928:0x00000000
    
    	   [C66xx_0] ********************************
    *********** CONSUMER ***********
    ********************************
    WARNING: Please ensure that the CONSUMER is executing before running the PRODUCER!!
    Debug: Waiting for module reset...
    Debug: Waiting for module local reset...
    Debug: Waiting for SRIO ports to be operational...  
    Debug: SRIO port 0 is NOT operational.
    Debug: SRIO port 1 is NOT operational.
    Debug: SRIO port 2 is NOT operational.
    Debug: SRIO port 3 is NOT operational.
    Debug:   Lanes status shows lanes formed as four 1x ports
    Debug: AppConfig Tx Queue: 0x2a0 Flow Id: 0
    Debug: SRIO Driver Instance 0x@00849120 has been created
    Debug: Running test in polled mode.
    Debug: SRIO Driver handle 0x849120.
    
    
    [C66xx_1] ********************************
    *********** PRODUCER ***********
    ********************************
    WARNING: Please ensure that the CONSUMER is executing before running the PRODUCER!!
    Debug: Waiting for module reset...
    Debug: Waiting for module local reset...
    Debug: Waiting for SRIO ports to be operational...  
    Debug: SRIO port 0 is operational.
    Debug: SRIO port 1 is operational.
    Debug: SRIO port 2 is operational.
    Debug: SRIO port 3 is operational.
    Debug:   Lanes status shows lanes formed as four 1x ports
    Debug: AppConfig Tx Queue: 0x2a0 Flow Id: 0
    Debug: SRIO Driver Instance 0x@00849030 has been created
    Debug: Running test in polled mode.
    Debug: SRIO Driver handle 0x849030.
    
    
    Latency: (DIO_NW, 5.000GBaud, 1X, tab delimited)
    Core	Lanes	Speed	Conn	MsgType	PktSize	NumPkts	MnLCycs	AgLCycs	MxLCycs
    1	1	5.000	B-E-B	DIO_NW	4	100	290	290	324
    1	1	5.000	B-E-B	DIO_NW	8	100	290	290	305
    1	1	5.000	B-E-B	DIO_NW	16	100	295	295	310
    1	1	5.000	B-E-B	DIO_NW	32	100	308	308	323
    1	1	5.000	B-E-B	DIO_NW	64	100	326	326	341
    1	1	5.000	B-E-B	DIO_NW	128	100	362	362	382
    1	1	5.000	B-E-B	DIO_NW	256	100	434	434	436
    1	1	5.000	B-E-B	DIO_NW	512	100	542	542	562
    1	1	5.000	B-E-B	DIO_NW	1024	100	758	771	778
    
    [C66xx_0] Throughput: (RX side, DIO_NW, 5.000GBaud, 1X, tab delimited)
    [C66xx_1] Throughput: (TX side, DIO_NW, 5.000GBaud, 1X, tab delimited)
    [C66xx_0] Core	Lanes	Speed	Conn	MsgType	OHBytes	PktSize	Pacing	Thruput	PktsSec.	NumPkts	PktLoss	AgPCycs	AgLCycs	AgICycs	AgOCycs	Seconds
    0	1	5.000	B-E-B	DIO_NW	16	4	0	8.40	262536.09	1400000	No	3809	21	3788	0	5.33
    [C66xx_1] Core	Lanes	Speed	Conn	MsgType	OHBytes	PktSize	Pacing	Thruput	PktsSec.	NumPkts 	PktLoss	AgPCycs	AgLCycs	AgICycs	AgOCycs	Seconds
    1	1	5.000	B-E-B	DIO_NW	16	4	0	83.99	2624672.00	12600000	No	381	325	41	15	4.80
    [C66xx_0] 0	1	5.000	B-E-B	DIO_NW	16	8	0	16.80	262536.09	1400000	No	3809	21	3788	0	5.33
    [C66xx_1] 1	1	5.000	B-E-B	DIO_NW	16	8	0	167.98	2624672.00	12600000	No	381	325	41	15	4.80
    [C66xx_0] 0	1	5.000	B-E-B	DIO_NW	16	16	0	33.60	262536.09	1400000	No	3809	21	3788	0	5.33
    [C66xx_1] 1	1	5.000	B-E-B	DIO_NW	16	16	0	335.96	2624672.00	12600000	No	381	325	41	15	4.80
    [C66xx_0] 0	1	5.000	B-E-B	DIO_NW	16	32	0	67.21	262536.09	1400000	No	3809	21	3788	0	5.33
    [C66xx_1] 1	1	5.000	B-E-B	DIO_NW	16	32	0	671.92	2624672.00	12600000	No	381	325	41	15	4.80
    [C66xx_0] 0	1	5.000	B-E-B	DIO_NW	16	64	0	134.42	262536.09	1400000	No	3809	21	3788	0	5.33
    [C66xx_1] 1	1	5.000	B-E-B	DIO_NW	16	64	0	1343.83	2624672.00	12600000	No	381	325	41	15	4.80
    [C66xx_0] 0	1	5.000	B-E-B	DIO_NW	16	128	0	268.84	262536.09	1400000	No	3809	21	3788	0	5.33
    [C66xx_1] 1	1	5.000	B-E-B	DIO_NW	16	128	0	2687.66	2624672.00	12600000	No	381	325	41	15	4.80
    [C66xx_0] 0	1	5.000	B-E-B	DIO_NW	16	256	0	537.67	262536.09	1400000	No	3809	21	3788	0	5.33
    [C66xx_1] 1	1	5.000	B-E-B	DIO_NW	16	256	0	5375.33	2624672.00	12600000	No	381	325	41	15	4.80
    [C66xx_0] 0	1	5.000	B-E-B	DIO_NW	16	512	0	1075.35	262536.09	1400000	No	3809	21	3788	0	5.33
    [C66xx_1] 1	1	5.000	B-E-B	DIO_NW	16	512	0	10750.66	2624672.00	12600000	No	381	325	41	15	4.80
    [C66xx_0] 0	1	5.000	B-E-B	DIO_NW	16	1024	0	1587.90	193836.02	1000000	No	5159	21	5138	0	5.16
    
    [C66xx_1] 1	1	5.000	B-E-B	DIO_NW	16	1024	102	15875.97	1937984.50	9200000	No	516	324	177	15	4.75
    
    Latency: (DIO_NR, 5.000GBaud, 1X, tab delimited)
    Core	Lanes	Speed	Conn	MsgType	PktSize	NumPkts	MnLCycs	AgLCycs	MxLCycs
    1	1	5.000	B-E-B	DIO_NR	4	100	509	509	532
    1	1	5.000	B-E-B	DIO_NR	8	100	509	509	532
    1	1	5.000	B-E-B	DIO_NR	16	100	509	509	532
    1	1	5.000	B-E-B	DIO_NR	32	100	508	509	545
    1	1	5.000	B-E-B	DIO_NR	64	100	508	512	555
    1	1	5.000	B-E-B	DIO_NR	128	100	508	550	587
    1	1	5.000	B-E-B	DIO_NR	256	100	508	614	656
    1	1	5.000	B-E-B	DIO_NR	512	100	508	719	763

  • Also, is there a specific timing limit when you should run the 2 programs? I know I should run producer first then consumer. But like, how long do I have to wait after I start producer to run consumer?

    Thanks,
    Shang
  • Hi,

    Your SRIO throughput example log also show some error messages.

    Have you connected the two EVMs via BOC like below link on your test setup.
    processors.wiki.ti.com/.../DualEVM_BOC_quick_setup_guide_09SEPT13.pdf
    Please share your test setup image

    Thanks,
  • Hi Ganapathi,

    I just took a pic of my evm and breakout boards.

  • Hi,

    Your setup is correct. I am also using the same setup to test the SRIO IPC example on my EVM's with multiple times it is working fine.

    Thanks,