• State TI Thinks Resolved
  • Locked Locked
  • Replies 7 replies
  • Answers 1 answer
  • Subscribers 99 subscribers
  • Views 577 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Multiple Tasks accessing PCIE with Interrupts

Brian Marks
Prodigy 80 points

Other Parts Discussed in Thread: 66AK2E05

We are using the TI Keystone II 66AK2E05 to interface to with a Xilinx Virtex 7 FPGA, over PCIe.   When we read/write memory in the FPGA in a single task, everything works as expected.   When we have multiple tasks that are using the PCIe, we are seeing issues where data appears to be corrupted.   More specifically, we have a thread that is periodically reading blocks of data over PCIe (currently without DMA).   We then have a HWI service handler routine, which also reads data over PCIe.   We have observed that frequently the first several reads in the HWI service handler routine contains corrupted data.   What do we need to do to allow a task, and a HWI service routine to read/write memory over PCIe, to prevent data corruption on reads in the HWI?   It appears as if the PCIe core complex is in the middle of handling a non-posted read on the buss, and then the DSP gets a HWI, and then it attempts to read on the PCIe buss, and this puts the interface into a bad state.   Does a non-posted read on the buss, not block the CPU?

  • 0
    TI__Mastermind 28085 points

    Hi,

    Have you using legacy PCIe interrupt on your test? If yes, it support RC mode only.

    Have you configured the DSP as Root Complex? Have you using TI provide MCSDK PCIe example for your testing?

    Please go through below wiki for PCIe related FAQs and Resources.

    http://processors.wiki.ti.com/index.php/PCI_Express_(PCIe)_Resource_Wiki_for_Keystone_Devices



    Thanks,

  • 0 in reply to Ganapathi Dhandapani95
    Prodigy 80 points
    We have DSP configured as a Root Complex. The FPGA is the endpoint. We are using a single MSI interrupt (no Legacy interrupts).

    The PCIE driver we are using was based on the MCSDK PCIE project.

    The DSP ISR that we've written reads a status register in the FPGA and then posts a semaphore so a task can run. The status register has some bits indicating that various interrupts are enabled as well as some bits to indicate which interrupts have triggered. What we have observed is that we can sometimes make over 200 reads that incorrectly reads back an enable bit that should be set and then it eventually gets read correctly.
  • 0 in reply to Brian Marks
    TI__Mastermind 28085 points
    Hi,

    Thanks for your reply. Have you got the same result with multiple test?

    Thanks,
  • 0 in reply to Ganapathi Dhandapani95
    Prodigy 80 points
    Yes.

    We most often see the issue when tasks of different priority are interacting with PCIE. In general we have a handful of high priority tasks of the same level that are tasks running from semaphores that are posted by the main ISR. We have a lower priority task that is essentially a message processing task that we use to copy data out of the DSP and send to a PC application. The "bad" reads seem to always happen whenever this lower priority task runs. We have tried disabling interrupts within the low priority tasks but the bad PCIE read problem still occurs.
  • 0 in reply to Brian Marks
    TI__Mastermind 28085 points
    Hi,

    PCIe interrupt and priority does not effect the read/write data. I have tested the MCSDK test code between two devices with and without interrupt mode. It is working fine.

    Have you using PCIe switch on your setup? Multiple end points are connected on your setup?

    Thanks,
  • 0 in reply to Ganapathi Dhandapani95
    Prodigy 80 points

    Thanks for looking into it. We do only have 1 endpoint with no switch.

    We believe we've stumbled onto the underlying issue.  We think that it's related to the Xilinx core not being able to handle 16 or 8 bit reads and writes.  After going through our DSP code to make sure that only 32 bit read/writes are occurring the problem has gone away.  We are still trying to determine if we can fix the FPGA issue or not.

    We do hope to transition some of our block copies to/from the FPGA into the DSP's EDMA3 engine.  I do have an additional question about how EDMA3 works across the PCIE interface.  If the DMA engine is given source and/or destination addresses that are NOT 32 bit aligned will it handle that across PCIE with 16 and 8 bit transactions or will it do only 32 bit transactions?

  • 0 in reply to Brian Marks
    TI__Mastermind 28085 points
    EDMA also use the 32-bit transactions.