This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/TMDXIDK5718: Cannot find PCIe EP device on host

Part Number: TMDXIDK5718
Other Parts Discussed in Thread: AM5728

Tool/software: Linux

Hello,

This is a question about the PCIe endpoint on TMDXIDK5728 board but the forum doesn't have it as part number so I set TMDXIDK5718.

I'm using the PCIe endpoint in enpoint mode and want to test with the Linux endpoint framework driver. To use the PCIe endpoint in endpoint mode I followed the docs and made a modified PCIe cable:

http://software-dl.ti.com/processor-sdk-linux/esd/docs/latest/linux/Foundational_Components_Kernel_Drivers.html#pcie-end-point

http://processors.wiki.ti.com/index.php/PCIe_CableMod

I connected the board with a Linux PC via the modified PCIe cable, followed the instructions in the doc, and then rebooted the PC, but there's no new PCIe device found (lspci shows no new device). On the board Linux system when I bind the endpoint function driver to endpoint controller driver there're warnings about BAR4/5 not successfully set, but it's expected because there're only 4 inbound windows according to the device tree file. No other warnings/errors shown.

What could be the problem? Is this feature only for two same board connecting each other, not for PC with the board? Or maybe it's the modified cable issue? I cannot get the exactly same cable as the wiki, so I bought a PCIe 1x male-to-male cable, and removed all 12V, 3.3V, clk+/- pins on one side.

Thanks!

  • Hi, 

    In TI's PCIe EP User's Guide, the EP test was done by connecting 2 AM57x EVM through a modified cable. The purpose of using the modified cable is not having both EVMs sending power and clock signals to the other end, and each EVM uses its own clock. This is ok when both RC and EP are AM57x EVM because the clock are in sync. When connecting to a PC, Windows or Linux, or any non-AM57x device, the EP needs to get the clock from the RC. 

    Please refer to the following thread on the discussion on how to sync the clock between a PC and AM57x EP.

    https://e2e.ti.com/support/processors/f/791/p/653443/2404690#pi320966=2

    Using modified cable, you cut off the clock pin which definitely won't work.

    Rex

  • Hi Rex,

    I read that thread but it seems there's no solution provided. If I keep the clock pins on the cable, will it work? The AM57x device will get the clock signals from RC and not send clock signals to RC? If this doesn't work, I guess the only option is using two AM57x devices connecting each other, since there're hardly some PCIe settings available on a normal PC BIOS?

    Thanks.

  • Hi, Zhangxin,

    In the EP test setup using AM572x EVMs, if both EVMs have clock going through the cable, the board will be damaged. That's why the clock pin was cut in our setup. 

    Having the RC clock going through the cable is not enough and that is what was discussed in the thread. The bottom line is EP needs to take clock from RC. This involves disabling the internal clock input to PCIE_SS and programing ACSPCIE to be in RX mode. 

    Rex

  • To be clearer, the PCIE_SS shouldn't use current internal clock source, but take the external clock from RC through LJCB_CLKP/N by programming ACSPCIE in RX mode.

    Please refer to section 26.4 PCIe PHY Subsystem of the AM5728 TRM, SPRUHZ6J, and take a look at Figure 26-17.

    Rex

  • Thanks Rex for the additional information. It seems more works need to be done which I'm not familiar with, I'll use two TMDXIDK5728 boards with the modified cable for this feature. Thank you for your kind help!

  • Hi Rex,

    I've got another TMDXIDK5728 board, and tried to connect the 2 boards with the modified cable. I built different Linux kernel images for the 2 boards, one enabled DRA7XX_HOST one enabled EP, followed the steps (http://software-dl.ti.com/processor-sdk-linux/esd/docs/latest/linux/Foundational_Components_Kernel_Drivers.html#pcie-end-point) in EP, then booted the board of rc mode. But from the host board only the PCIe controller device (0x104c:0x8888) is found, no endpoint device. What do you think may be the problem or there're more configurations to be done? Thanks!

  • Hi, Zhangxin,

    The words of "enabled DRA7xx_HOST" alerted me. You shouldn't need to do anything for RC side. The PSDK release has RC enabled by default. You only need to change DTS file to enable EP and rebuilt the kernel. Use the released kernel images for RC. Then follow the steps in the EP User's Guide to bring up EP. After EP is up, then bring up RC. There is a how-to video on this EP test. You may want to take a look at it if you haven't.

    https://training.ti.com/am57x-pcie-ep-demo-processor-sdk-linux

    Rex

  • Hi Rex,

    I used the pre-built kernel as host, it cannot find the endpoint device too. The video used the old release so the kernel config is not the same as latest PSDK, and another difference is a AM57xx-EVM board is used as EP with a miniPCIe-PCIe adapter, while I used 2 TMDXIDK5728 boards.

    Here're the configurations of my setup:

    2 TMDXIDK5728 boards, modified PCIe X1 cable with 12V/3.3V/clk+- pins disabled;

    The PSDK used is the latest 05.03.00.07, kernel 4.14.79, when rebuild the kernel the used config is tisdk_am57xx-evm_defconfig, dtb is am572x-idk.dtb;

    Followed the steps on EP (http://software-dl.ti.com/processor-sdk-linux/esd/docs/latest/linux/Foundational_Components_Kernel_Drivers.html#pcie-end-point), the endpoint controller folder in configfs is there, and all settings didn't show additional errors. After echo 1 to controller start, boot the RC board;

    on RC board console lspci shows only PCIe controller 0x104c:0x8888, dmesg shows this line, not certain it's expected or not: "dra7-pcie 51000000.pcie: phy link never came up".

    Xin

  • Hi, Xin,

    We run PCIe EP nightly test for each night's build. Though it is run on the other EVM, DRA7x-EVM, not the AM57x-EVM, but they are the same family SOC. AM57x is based on DRA7x. It's hard to understand why it doesn't work for you.

    As development moving along, some steps needed during the video was shot are included as default in the later release, such as enabling the EP mode, and epf test through menuconfig. Some are still needed. I assume for the EP, you have rc status set to disabled, and ep status to okay in the dts file.

    You won't see EP if you get phy link never came up. The handshaking with EP didn't complete.

    Rex

  • Hi, Xin,

    We have a user in another thread who successfully tested PCIe EP mode. Please refer to his original post in 

    https://e2e.ti.com/support/processors/f/791/t/811071

    and he stated the EP test was successful and is able to move forward in 

    https://e2e.ti.com/support/processors/f/791/t/818701

    Please be sure to follow the steps as stated in the document. If your issue is resolved, please click "Resolved".

    Rex

  • Hi Rex,

    I see you mentioned this in both threads:

    "the EP should take clock from RC. So, the PCIE_SS shouldn't use current internal clock source, but take the external clock from RC through LJCB_CLKP/N by programming ACSPCIE in RX mode.

    Please refer to section 26.4 PCIe PHY Subsystem of the AM5728 TRM, SPRUHZ6J, and take a look at Figure 26-17."

    If the another TMDXIDK5728 plays the RC role, this is not needed as the EP using the internal clock which is same as the RC clock?

    Xin

  • Hi, Xin,

    That is correct. When both RC and EP are TI AM57x  EVMs, each can run its internal clock and they are in sync.

    Rex

  • Hi,

    To who finds this thread for the same question, despite the clock/power is disconnected on the cable, my colleague swapped Tx and Rx of the cable then the board as RC can find the board as EP connected.