This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Bring up PCIE link after reset to EP

Other Parts Discussed in Thread: TMS320C6678

The system has TMS320C6678 as PCIE RC, the EP is implemented in FPGA device. Was able to bring up the link successfully after power up(using Code composer). After reset the FPGA(DSP does not got reset in this case), the PCIE link will go down. I have try the following to try the link back up:

1) write to link status control register bit 5 to enable retraining, the link seem to stay down

2).run the same PCIE init sequence, PCIE_InitPcieRc as boot code to init both RC and EP, the link seems to come up, but failed to do any configuration cycle.

My suspicion is that need to shut down the PCIe link in the DSP gracefully before asserting reset to the EP.

What is the correct sequence to bring the PCIE link back to operation after resetting the EP device?.

 

Thanks

  • Welcome to the TI E2E forum. I hope you will find many good answers here and in the TI.com documents and in the TI Wiki Pages (for processor issues). Be sure to search those for helpful information and to browse for the questions others may have asked on similar topics (e2e.ti.com). Please read all the links below my signature.

    Moved this thread over keystone forum. We will get back to you on the above query shortly. Thank you for your patience.

  • Hi Andy,

    Please go through below wiki for PCIe related FAQs and Resources for PCIe link issue,

    processors.wiki.ti.com/.../PCI_Express_(PCIe)_Resource_Wiki_for_Keystone_Devices

    Thank you.
  • Hi,

    Please go through the Power sequencing steps on below PCIe FAQ wiki for bring the PCIE link back to operation after resetting the device.
    processors.wiki.ti.com/.../PCI_Express_(PCIe)_Resource_Wiki_for_Keystone_Devices

    Thanks,
  • HI Ganapathi,

    Thanks for the info. I did not find any info from the Wiki page about re-train the link and bring it back up to normal operation. The power sequencing did not get involved in this case. Like I mentioned before, the link was able to up and operational before asserted a reset to the end point device which cause the link to go down. This reset has nothing to do with power, should not affect the DSP except the PCIE link now is down. I changed re-training sequence a little bit. Before I pull the reset to FPGA, write to clear bit 0 LTSSM_EN  in DSP PCIE command status register(in PCIE application space), this will disable the link first. After the reset to FPGA is dis-asserted. write to set bit LTSSM_EN, this should force link to re-training again. Read to the Debug 0 register, did indicate the link came up, but again, was not able do configuration read/write the PCI EP device, the PCIE_UNCERR register has FCP_ERR_ST bit set, flow control protocol error.

    I am running the code with a emulator, if I halt the code, and re-run the boot code which re-init the everything include the PCIE RC and PCIE EP, basically got the same error. But I do a system reset from the emulator, and then re-run the boot code, then it seems OK. the FPGA (PCIE EP) did not got re-pumped in this case ,that kind of imply the problem is not the FPGA side. 

    Again, your help is greatly appreciated.

    Thanks 

  • Hi,

    C66xx devices does not support for PCIe hot plug. If you reset the DSP means, you need to reset the both RC, EP power and re-initialize the PCIe registers to bring the PCIE link back to operation.

    TI doesn’t provide PCIE RC enumeration example/driver source. Normally the RC runs some operating system that performs the enumeration. In the MCSDK PCIe sample example, two DSP EVMs are used to test the PCIe driver. DSP 1 is configured as a Root Complex and DSP 2 is configured as End Point. Once the PCIe link is established, the following sequence of actions will happen:
    1. DSP 1 sends data to DSP 2 - DSP 2 waits to receive all the data
    2. DSP 2 sends the data back to DSP 1
    3. DSP 1 waits to receive all the data
    4. DSP 1 verifies if the received data matches the sent data and declares test pass or fail.

    This example is used for validating the MCSDK PCIe driver. It does not directly support for DSP to FPGA PCIe test.

    Thanks,
  • Hi Ganapathi,
    With this sequence, it seems to close back to normal.
    -After releasing FPGA(PCIE EP)reset(again, the FPGA reset will cause EP to lose PCIE link, nothing else on the RC side), Disable the PCIe power domain(as well as the PCIe module)
    -Ran thru the same PCIE init sequence as boot code(the init code re-enable the power domain).
    Everything seems to be normal, able to access to the EP device without any error, but somehow, the device ID of the RC is now read back as 0x8888, it was 0xB005 before this sequence. Any idea why getting different device id now? (Our boot code is similar to the TI example boot code, we did not use IBL)

    Thanks
  • Hi,

    PCIe device ID and vendor ID fields are swapped in CSL register maps. These CSL macros values are corrected in MCSDK 2.1.3.x package. But i don't know the exact timeline for release of this MCSDK version. Better to correct the CSL macro values on your code, refer below similar thread for CSL macro correction:
    e2e.ti.com/.../385456

    Have you testing the DSP PCIe example code on No-Boot mode? What version of C6678 silicon chip you using? 1.0 or 2.0

    Refer Usage Note 1 on C6678 silicon errata document.
    The PCIe vendor ID / device ID field reset value is incorrect if PCIESSEN (PCIe enable/disable) pin is low at boot time (disable). If PCIESSEN pin is low at boot time, the vendor ID / device ID PCIe register needs to be set by software with the proper values. The vendor ID should be 0x104C. The device ID should be 0xB005. In addition, the register should be set to the correct value before enabling the link training sequence. Note: When PCIEEN is set, ROM will set the vendor ID / device ID PCIe register with the proper values. Therefore, software does not need to do it in this case.

    Thanks,
  • Hi Ganapathi,
    I am not using CSL macro, instead I do direct memory read to location( 0x2180000 + 0x1000)for vendor ID and device ID. Again. During this reset sequence, there is not "reboot", Just power down the PCIE domain, and then re-initialize just the PCIE domain. The vendor ID is correct, just the device ID is different this time.
    During a normal boot(for example from power up), how the device ID got set? from your email it seem that is feed from ROM, it is by hardware? probably not by software, since I did not see that in our boot code?

    Thanks
  • Hi,

    Better to correct the CSL macro values on MCSDK example code and use the rebuild binary for your testing. If you still got the same result, please share the project zip file. i will try to reproduce your issue and try to resolve.

    Thanks,
  • Andy,
    Would you mind to share a little bit more on your solution? I am interested to reboot FPGA while DSP is running, so the situation is similar, I have to reestablish the link. Have no progress so far.
    Thanks in advance.