We have a working PCIe example using the C6678 EVM and an FPGA EVM connected through a custom PCIe backplane. Link training works fine, and we are able to write / read data to / from the FPGA from the DSP.
We are now trying to run the same example over a fiber optic PCIe link (Finisar Laserwire). We have two of custom PCIe backplanes (both verified) - one with the DSP EVM and a Laserwire adaptor board, the other with the FPGA EVM and a Laserwire adaptor board.
With this setup, the DSP never gets past the DETECT_QUIET ltssm value during link training.
Is there a way to force the DSP's PCIe state machine past the DETECT_QUITE state?
Any trouble-shooting suggestions will be greatly appreciated.
~ Andrew.
Andrew,
1. As you said, the two PCIe backplanes are verified, what about the two Laserwire adapters and the Laserwire link between them?
Do they have the good signal quality over the Laserwire link please?
2. Have you checked the PCIe reference clocks provided to both DSP EVM and FPGA please?
Do they have good quality (and same frequency) on those two separate backplanes?
I think you verified the DSP and FPGA are working in the single backplane testing. Then any new components in the two backplanes testing will be in question first.
Hope it helps.
Sincerely,
Steven
------------------------------------------------------------------------------------------------------------
Please click the Verify Answer button on this post if it answers your question.
Steven,
I work with Andrew. All the hardware is verified without the LaserWire link and hardware, less the LaserWire, is verified with the LaserWire link, i.e. PCIe referenc clocks. I believe we started with your software example. The problem we are having specifically is that the DSP PCIe controller sends out a pulse on both TX+ and TX-. It is looking for a path back to common mode and I do not believe the LaserWire is satisfying that so link negotiation fails here and retries. We have compared scope traces on TX+ & TX- with end-point removed, FPGA eval board as end-point installed, and LaserWire installed. The LaserWire appears as an open termination to the DSP PCIe controller. With the PCIe connected electrically, link negotiation performs this pulse once, is satisified, and moves on to the next state. We have verified the LaserWire physically works by temporarily running the PCIe clock over it. The DSP PCIe controller does not detect the LaserWire, which its specs state has a 100 Ohm termination and supports PCIe. The problem I fear is that the LaserWire does not have a balanced termination back to ground - the required PCIe common mode termination.
So I have tried to justify "why" we want to do the following as I don't believe the LaserWire is ever going to satisfy the detect test.
We are hoping that there is some way we can tell the DSP PCIe controller to assume that a valid receiver exists and force it to bypass this first step, receiver detection, in link negotiation as we are confident the LaserWire will carry data, but we have no way to verify this without bypassing receiver detection.
Joel,
I see your point. The normal link training states of PCIe are Detect -> Polling -> Configuration -> L0 (normal operation).
If you would like to skip the first "Detect" step and move forward, you can give a try on one register in C66x PCIe module as PL_FORCE_LINK (0x21801708). There is one "FORCE_LINK" field to force the link to the state specified by "LINK_STATE" field.
Based on the LTSSM States table in Appendix. A of C66x PCIe user guide, the POLL_ACTIVE state is coded as 0x02. If your current step is stuck in "DETECT_QUIET" and "DETECT_ACT" stages, you can set the "LINK_STATE" field to be 0x02 and set 1 in "FORCE_LINK" field to see if PCIe module could be forced to Polling Active status and move forward.
I do not think we can just force to link to "L0" status since there are lots of negotiation to be done between PCIe link peers in Polling and Configuration states, such as data rate, lane number, polarity, etc.
It may need both ends (C66x PCIe and FPGA PCIe ports) to be forced to the same link training status.
Hope it helps. But I am not sure if it could work. Otherwise, we may need to think of how the LaserWire could be compatible with PCIe link requirement since the LaswerWire spec says so.
Hi Steven,
By forcing both the DSP PCIe controller and the FPGA PCIe core to skip their DETECT state, the link is up and running over the LaserWire.
Many thanks!
Joel.