TMS320C6678: evmc6678 not enumerated after PC restart (it just got enumerated at a certain time)

malek alashiri

Part Number: TMS320C6678

hello
PC can enumerate evmc6678 for first time, but after PC restart evmc6678 won't get enumerate anymore. also we have one Xilinx ZCU102 and one SAMSUNG SSD device. Xilinx ZCU102 can't enumerate SAMSUNG SSD DISK in first time. we need to restart Xilinx board once for SAMSUNG SSD to get enumerated (view it using lspci -n). our main goal is to feed evmc6678 into Xilinx board but Xilinx can't enumerate evmc6678 at all.

in this stage we reach to this conclusion:
Xilinx first stage bootloader finish it's enumeration process too fast so neither SAMSUNG SSD nor evmc6678 can't get enumerated. but SAMSUNG SSD has the ability to re-enumearte after Xilinx board reset while evmc6678 can't do that (we conclude this from the fact that PC can't enumerate evmc6678 after restart.)

it seems that SAMSUNG board is Always ready to get enumerated but this isn't true about evmc6678.

in all of TI code for PCIE initialization, there is always this pattern (both in IBL code and example in sprabk8.pdf)

(a) lock pll
(b) disable training
(c) write some settings
(d) enable training and wait for training to end.

it seems that whole problem originate from stage (d). in other word, once DSP reach stage (d) if training process was not successful, it will get blocked in a while loop and host reset have no effect in breaking this loop.

we also try to use APP_RETRY_EN bit in CMD_STATUS which wasn't successful. here is how we did this:

by assuming that FPGA finish its enumeration process before DSP to be ready, we enable APP_RETRY_EN in CMD_STATUS so that every request from FPGA will get RETRY response. then we add 6 sec idle before enabling training process. this way after fpga boot we give it one reset in less than 6 sec. what we expected was for FPGA to get blocked by RETRY response from DSP and after DSP Activate it's training FPGA can enumerate it. but this won't work either.

so can anyone suggest any solution to this problem? how can SAMSUNG SSD board be always ready for training but evmc6678 can't ?

(all of the above tested by both TI IBL and our custom IBL)

over 7 years ago

0 Pavel Botev over 7 years ago

TI__Guru**** 170625 points

Hi Malek,

Regarding C6678 device, which RTOS version you are using on it? The latest is v5.00:

www.ti.com/.../processor-sdk-c667x

Regards,
Pavel

0 malek alashiri over 7 years ago in reply to Pavel Botev

Intellectual 680 points

hello
for IBL i used pre-build binary of mcsdk_2_01_02_06 which by my knowing is latest version of mcsdk. we didn't use RTOS.
and for our custom code we used sprabk8 & sprugs6d as reference. using of APP_RETRY_EN was suggested in sprugs6d.

0 lding over 7 years ago in reply to malek alashiri

TI__Guru* 95265 points

Hi,

Can you clarify this is TI 6678 EVM or your own 6678 board?

If the IBL code on DSP side runs too fast, the Linux machine may sends out a hot reset after the enumeration, When receiving a PCI-Express (PCIe) Hot Reset request during the PCIe boot process, the PCI-Express Subsystem (PCIESS) will disable the Link Training and Status State Machine (LTSSM) and suspend the PCIe bus in the Detect.Quiet state. With the LTSSM disabled, there will be no subsequent transition from Detect.Quiet to Detect.Active and the PCIe boot process will fail. If this is the case, you can connect JTAG/CCS to check 0x2180_0004 bit 0, LTSSM_EN. If it is 0, that means the LTSSM is disabled due to hot reset.

If the IBL code on DSP side runs too slow, as the host Xilinx runs too fast for enumeration, can you confirm that LTSSM is still 1 in DSP side? Does Xilinx run Linux? Do you have way to slow down the enumeration process to make sure the DSP side is ready?

Another way, may be you can create an customized IBL on C6678 side, just do the PCIE workaround: setup clock selection, PLL, PCIE registers and enable training. This IBL will be smaller and boot much faster to meet the enumeration window. We did this test many years ago, and due to the IBL ran too fast, and it first got enumerated by a Linux PC, then got hot reset. We had to add some delay (about 0.5 to 1 seconds) in front of the IBL code. This give you some confidence that if your IBL is smaller, it should be ready for Xilinx enumeration.

Regards, Eric

0 malek alashiri over 7 years ago in reply to lding

Intellectual 680 points

hello lding
thanks for reply
1.
we are using TI 6678 EVM board
we tested both TI IBL and our Custom IBL. our code is faster than TI IBL because :
TI Code Size = 53 Kb while our IBL code Size = 18 KB
and also we are using 400KHz bus for I2C rather than 20KHz bus.
but both our IBL & TI IBL show same behavior. i means they both get enumerated on PC but not on Xilinx board.

2.
bit 0 of 0x2180_0004 (CMD_STATUS) is always 1 (CMD_STATUS = 7) . even after we reset FPGA board it remains 1 so it means that training isn't disable. we also checked Debug0 and Debug1 Values.
bit 0-4 of Debug0 is always 3 which means DSP have stopped in POLL_COMPLIANCE state.(i don't know what exactly it means so i hope it will be useful info.)
value of Debug1 is 0x08000000 most of the time and sometimes turn to 0x08100000. based on Debug1 data we assume that:
a. LINK is NOT disabled -> bit 30 is equal to 0
b. LINK is NOT performe Training -> bit 29 is equal to 0
c. bit 27 which is TRAINING_RST_N is set, we don't know its meaning.

so obviously link Training is't disabled BUT based on Debug1, link training isn't active too. is this information useful?
we still don't understand what's going on?

3. Xilinx run linux too but its link training run inside "first stage boot loader" which execute before linux. i'm not sure if we could slow down fpga but once i've confirmed it and test it, i will let you know

0 lding over 7 years ago in reply to malek alashiri

TI__Guru* 95265 points

Hi,

In Xilinx case, do you use lane 0 for connection? Can you check 2.3.1.2 PCIe SerDes Status Register (PCIE_SERDES_STS) 0x0262015C, did you see the lane lock or signal loss?

What is the reference clock of Xilinx? Is it fixed 100MHz? Do you pass the Xilinx clock to C6678 EVM and in the IBL code you controlled that signal coming from AMC edge connector into C6678?

What about if you separate PCIE reference clocks? I thought that Xilinx should be using a fixed 100MHz clock, not spread spectrum. Is possible that you just put the 6678 EVM in no boot mode, and use CCS/JTAG to load the existing PCIE sample came with Processor SDK or MCSDK, run the C6678 in EP mode (the clock come from the on-board crystal, not from Xilinx side) first. Then run the Xilinx to see if you can enumerate C6678?

Regards, Eric

0 malek alashiri over 7 years ago in reply to lding

Intellectual 680 points

hello
1. value of PCIE_SERDES_STS is 0x309 which means we have "loss of signal detect" on both lane.

2. xilinx clock is 100 MHz and by default it is set for SSC and our configuration was common clock too.(we got enumerated on PC)

3. we separate PCIE power from PCIE slot and set FPGA clock to normal 100 MHz, then we set our setting for separate clock configuration inside DSP code(we also switch evmc6678 FPGA setting to feed on-board clock to DSP) . we first power up DSP and wait for its pll to get lock and start its training process. then we power up FPGA, the interesting thing is, as soon as FPGA turned on, we enter POLL_COMPLIANCE state.

0 lding over 7 years ago in reply to malek alashiri

TI__Guru* 95265 points

Hi,

Thanks for explaining the Xilinx is also SSC clock by default and your usage of common clock is valid. Also for the testing of separated clock.

For #1, 1. value of PCIE_SERDES_STS is 0x309 which means we have "loss of signal detect" on both lane. If you have signal loss on Serdes level, there is way the PCIE can work. Can you resolve the hardware issue first? For 6678 EVM connected into a Linux PC, I understand that you used an AMC-PCIE adapter card. Please cross check in the working case, the 0262_015c = 0x1.

When you use Xilinx with C6678 EVM, how they are physically connected? And how Xilinx and Samsung SSD are connected?

Regards, Eric

0 malek alashiri over 7 years ago in reply to lding

Intellectual 680 points

hello
1. on Linux PC, i read 0x201 for value of PCIE_SERDES_STS, which means we have loss of signal on "lane 1". something i forget to mention is that on Linux PC we are always in 1x , 2.5 Gbps mode. even thought we set evmc6678 to work on x2, 5 Gbps mode.

2. we are using AMC-PCIE adaptor card to connect evmc6678 to Xilinx board. Xilinx board supports 4-lane and have x8 slot on it. i don't know what you mean by " how they are physically connected? "

3. i understand that reason we are in 1x mode on linux PC is loss of signal on lane one. but what you mean by "If you have signal loss on Serdes level, there is way the PCIE can work" ?

4. how it is possible for same board to show different behavior on two system, i mean why we have loss of signal on both lane on xilinx board and only loss of signal on single lane on Linux PC. if this is a hardware problem how can i check it?

0 lding over 7 years ago in reply to malek alashiri

TI__Guru* 95265 points

Hi,

For #2, I just wanted to understand how the boards were connected. Now it is clear to me. 6678 EVM -----AMC to PCIE adapter ---- PCIE X8 slot ----- Xilinx FPGA.

For #3, sorry typo, I meant "If you have signal loss on Serdes level, there is NO way the PCIE can work"

For #4, Is the HW connection in good contact? I am not a HW person, maybe you can measure the receive eye signal at the C6678 side?

Regards, Eric

Processors

Processors forum

TMS320C6678: evmc6678 not enumerated after PC restart (it just got enumerated at a certain time)