This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TMDSEVM6657: Trouble Pushing PCIe Data from FPGA EP to DSP RC

Part Number: TMDSEVM6657

Hi,

We are adding PCIe to our design to facilitate high bandwidth data between a TI DSP and Xilinx FPGA.  In order to prototype the design, we're using a TMDSEVM6657LS eval board in RC mode connected to a Xilinx KCU116 Kintex Ultrascale+ board in EP mode.  The DSP code is based on the PCIe example design using PDK C665x V2.08.  The PCIe link is Gen2 x2.

The link initializes properly and both sides report 'link up'.  Furthermore, I can successfully read and write FPGA memory from the DSP.  The problem is pushing data from the FPGA to the DSP.  If I send a Requester Request TLP with one data word from the FPGA, I get no indication at the DSP that anything has happened.  If I send a TLP with two data words from the FPGA, I get a fatal error indication at the DSP.  No data is received in my Rx buffer in either case.  I have scoured the TI documentation, TI forums and internet in general and have have seen many interesting and relevant articles, but none points to a cause or even changes my symptoms.

Here's my general DSP setup:

CMD_STATUS = 0x07 // IB & OB address translation enabled, LTSSM enabled

STATUS_COMMAND = 0x00100146 // enable response to memory accesses

MEMSPACE = 0x00900080 // points to 1MB L2 SRAM, 32 bit addressing

PREFETCH_MEM = 0x00900080 // same as MEMSPACE

BAR1 (mask) = 0x0FFFFFFF

BAR1 = 0x90000000 // arbitrary inbound PCIe address

IB_BAR0 = 1 // use BAR1

IB_START0_LO = 0x90000000

IB_START0_HI = 0 // 32-bit addressing

IB_OFFSET0 = 0x00863C00 // address of my Rx buffer in L2 SRAM

On the FPGA side, bus master capability has been enabled.  While I cannot see the actual serial data generated by the FPGA, the Requester Request TLP for a two word write to 0x90000000 looks correct.

Any suggestions as to what might be wrong with my configuration would be greatly appreciated.

Thanks,

Stuart

  • The team is notified. They will post their feedback directly here.

    BR
    Tsvetolin Shulev
  • Hello!
    First of all, why don't you try with prefetching disabled?
    Next, I did not get what requester request really is, but as we speak about data supposed to land to DSP's memory, I guess that was write request from FPGA side. If so, PCIe write is outside caching subsystem, and probably you need explicit invalidation of cached buffer range before trying to examine it. It looks extremely strange, that you see errors on two-DWORD TLP. One thing I could suggest is to monitor FPGA's side to see whether TRN interface was operated properly, and monitor FPGA status registers for the same errors. Againg, having alive exchange from DSP and broken from FPGA looks not normal.
  • Hi rrlagic,

    Thanks for your reply.  I had previously tested the system with prefetch disabled and it made no difference.  As for caching on the DSP side, I'm only using L2SRAM so there is no L2 caching and I have invalidated the L1D cache line.  On the FPGA side, I see no error indication and no change in configuration register values.  I have verified the Bus Master Enable bit is set in the FPGA configuration register.

    Stuart

  • Hi,

    IB_OFFSET0 = 0x00863C00 ======> please use a global address like: 0x10863c00 instead of local address.

    Regards, Eric
  • Did this work for you?

    Regards, Eric
  • No, it didn't seem to make any difference.  However, I switched from a bare-bones PCIe IP block in the FPGA to a full featured PCIe/DMA block and now everything works.  The big difference is now, instead of generating the PCIe descriptors in my own RTL, the IP block generates them.  While I could never find anything wrong with the descriptors generated in my code, I have to believe they were in error based on the new results.

    Thanks for your help.