This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TMS320C6678: DSP Stack gets crashed while running Ethernet application

Part Number: TMS320C6678
Other Parts Discussed in Thread: SYSBIOS

Hi,

We are using 4 DSP in our customized board. 

We are running an Ethernet application for all DSPs. In which, about 1400 bytes of data are continuously transferring between Host PC & DSPs. Our application runs fine for about a few hours. After that, DSP3 alone gets crashed and unable to ping from the host PC. Pls, find below the crash log obtained from CCS.

Going to receive command

No Route exists for
::
18361.303 PBM_free: Invalid Packet
18361.303 mmFree: Double Free
18361.303 mmFree: Double Free
18361.303 mmFree: Double Free
A0=0x1 A1=0x2
A2=0x0 A3=0x8bb091e4
A4=0x0 A5=0xffffffff
A6=0x86402404 A7=0x0
A8=0x8bb32b00 A9=0x0
A10=0x8bb2a000 A11=0x0
A12=0x8bb26150 A13=0x0
A14=0xf A15=0x0
A16=0x864023da A17=0x0
A18=0x864023d0 A19=0x0
A20=0x20 A21=0x0
A22=0x0 A23=0x0
A24=0x0 A25=0x1
A26=0x0 A27=0x2005020
A28=0x0 A29=0x6a012
A30=0x40700000 A31=0x200
B0=0x0 B1=0x0
B2=0x15000102 B3=0x0
B4=0xc219705 B5=0x0
B6=0xc219705 B7=0x6c
B8=0x73 B9=0x3a
B10=0x8bb4c600 B11=0x0
B12=0x8bb4c600 B13=0x0
B14=0x0 B15=0x864026e8
B16=0x30 B17=0x8640264c
B18=0x8bb420a8 B19=0x80
B20=0x0 B21=0x2e5
B22=0xf B23=0x0
B24=0x0 B25=0x3000
B26=0x3000 B27=0x0
B28=0x0 B29=0x3
B30=0x0 B31=0x86402690
NTSR=0x10204
ITSR=0xf
IRP=0x8bb2a000
SSR=0x0
AMR=0x0
RILC=0x0
ILC=0x0
Exception at 0x0
EFR=0x2 NRP=0x0
Internal exception: IERR=0x1
Instruction fetch exception
ti.sysbios.family.c64p.Exception: line 248: E_exceptionMin: pc = 0x00000000, sp = 0x864026e8.
To see more exception detail, use ROV or set 'ti.sysbios.family.c64p.Exception.enablePrint = true;'
xdc.runtime.Error.raise: terminating execution

 

Please give your suggestion to debug this issue.

Regards,

Kartik.

  • Hi,

    Can you confirm if you used any TI software, like NDK or PA for Ethernet applications? Are all the 4 DSPs running the same application? All on core 0 for each DSP or multicore application? Will other DSP crash as well or only DSP3 crash every time? If only DSP 3, what is the difference between this DSP and the others?

    Regards, Eric 


  • Hi Eric,

    Thanks for the response.

    Can you confirm if you used any TI software, like NDK or PA for Ethernet applications?

    Yes, We're using:
                                  ndk_2_21_01_38,
                                  pdk_C6678_1_1_2_6,
                                  mcsdk_2_01_02_06.

    Are all the 4 DSPs running the same application?

    Yes, all the 4 DSPs running the same application.

    All on core 0 for each DSP or multicore application?

    Yes, Core 0 for each DSP.

    Will other DSP crash as well or only DSP3 crash every time?

    No, Only DSP3 gets crashed after some hours every time.

    If only DSP 3, what is the difference between this DSP and the others?

    There's no difference among the DSPs.

    Thanks & Regards,

    Kartik

  • Hi,

    There are some tips how to debug SYSBIOS issues:

    • video: 

    The NRP is 0x0. That means your program has branched to address 0x0, which should not happen. The most likely reason for that is either a corrupted stack or a too small stack.

    And B3 is 0x0, this is the return pointer, it is also wrong.

    You may find out the register info from the C66x CPU and Instruction set Reference Guide, which can be found on the product page.

    Also, why it is always happening on DSP3, assuming you used the same HW and SW?

    Regards, Eric

  • Hi,

    Did you get a chance to narrow down the cause of crash? Is software or hardware related? Is the DDR3 stable? What if you put everything into L2 and MSMC?

    Regards, Eric

  • Hi Eric,

    No, we are still facing the same issue with DSP3.

    we were about to conclude that it might be the issue with DDR and discussed with some of our colleagues and then we found that this same kind of issue has been already discussed with you by our colleagues Vidya & Avinash.

    pls find the link below for the thread discussion,
    e2e.ti.com/.../3324120

    On that, they were declared that the board was working with reduced data rate of 800MHz (Clk rate = 400 MHz) and We also thought that would be solution for the issue.

    But we found that the board get failed after few hours even with above mentioned modifications.

    You asked them to share the gerber file for further debugging in layout level.

    could you share any FTP server procedure or official mail ID to share our gerber file?

    Thanks & Regards,
    Kartik

  • Karthik,

    Yes, I recalled that and I had worked on helping reducing DDR3 speed in IBL. My colleague helped to review the DDR3 board design issue. Let me add him into this e2e thread to see what the steps to receive customer files and what level of consultation we can provide.

    Regards, Eric 

  • Kartik,

    Layout reviews are beyond the support we can provide.  We have extensive documentation showing examples and providing routing rules to help you achieve success.  This document discusses the basics and links to documents and tools to complete this process: KeyStone™ I DDR3 interface bring-up Application Report (SPRACL8).  This document provides basis for our support model: Understanding TI’s PCB Routing Rule-Based DDR Timing Specification Application Report (SPRAAV0A).

    As discussed in the previous thread, your REG_CAL and PHY_CALC worksheets look correct.  Your observation that the DDR memory on a single DSP fails intermittently while the DDR on others runs robustly affirms that their is a layout issue.  As stated previously, I recommend that you recheck the spacing, impedance and reference plane recommendations.

    You may also need to look for EMI and EMC interference.  The most common issue found here is locating a DC-DC supply close to the DDR memory and routing.  The AC ripple currents in the ground paths near the power stage can affect the DDR interface if they are allowed to enter that area.  Similarly, the magnetic fields created in the power stage can induce noise on nearby routes - even those shielded by ground planes.

    Tom