TDA4VM: PCIE ep external reference clock

Hi Yongliang,

   For question #1, pls refer to below link for external clock source

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1004565/faq-tda4vm-tda4vm-dra829v-routing-pcie-reference-clock-externally

  For question #2,  you must install R3/R4 for external source connection. So you want to know that if  series connection capatitors are necessary?

It's reqiured by PCIE spec. You can refer to below link for explanation.

https://electronics.stackexchange.com/questions/285705/why-place-inline-capacitors-on-pcie-traces

Your answer contradicts the answer to my previous question（TDA4VM: PCIE reference clock externally - Processors forum - Processors - TI E2E support forums）, and I'm a little confused now.

1. When TDA4 is used as an ep device, does it need to follow this configuration when using an external clock?

In the previous question, I configured it like this, but I couldn't find the ep device under /sys/kernel/config/pci_ep/controllers, so I couldn't configure it normally.

so,what else can i do？

Pls list your step what you did

linux kernel version：linux-5.10.41+gitAUTOINC+4c2eade9f7-g4c2eade9f7

sdk version:ti-processor-sdk-linux-j7-evm-08_00_00_08-Linux-x86-Install.bin

I test PCIE with two EVM boards in external reference clock. My operation process is as follows:

1、/cfs-file/__key/communityserver-discussions-components-files/791/pcie_2D00_ref_2D00_clock_2D00_out_5F00_sdk8.diff already patch to 

phy-cadence-sierra.c

2、device tree change    k3-j721e-common-proc-board.dts    pcie0、1、2  rc is disable . ep is enable.As shown below

3、compile kernel and install to sd .

4、our two EVM connect like this

5、ep device dial like this

6、start EP EVM board 

use devmem2 verify these register ，it set successful。

but in /sys/kernel/config/pci_ep/controllers only has 2900000.pcie-ep，no 2910000.pcie-ep和2920000.pcie-ep

I can not continually config EP according to 3.2.2.10. PCIe End Point — Processor SDK Linux for J721e Documentation

In my flow of operations, I did not change the resistors and capacitors on the EVM. As shown in the figure below, in EP mode, you need to remove R1 R2 R5 R6 and add R3 R4 C1 C2

My doubt is whether the generation of devices under the /sys/kernel/config/pci_ep/controllers path requires an external reference clock？

In the startup process, the EP device is started first, after the configuration is successful, and then the RC device is started, so if I change the resistance and capacitance, when I start the EP device (the RC device has not started at this time)  Will the devices under the path /sys/kernel/config/pci_ep/controllers still be generated?

Yongliang, 

Below are some quick notes based on the discussion from yesterday:

1. Answers to your questions

    1). By default, the epf driver will use internal refclk for PCIe, even though the board modification routed host's refclk to the TDA4. To be sure, please read the SERDES_RST register for SERDES1, which is the SERDES used for PCIe1. 

     2). The series capacitors are needed on the refclk. But it should be the cause of observations you saw, where removing the refclk will cause epf device not present. To be sure you can check the SERDES's internal PLL lock status. Address offset should be SERDES1 base address + e000. 

please reply back if you can not find these registers in the TRM. I am not able to access the TRM presently.

2. Boot sequence if using TDA4 as EP

• Option 1: If you have the option to issue fundamental reset from host, send a fundamental reset to restart probing, after EP has booted. Note that PCI support three types of reset - fundamental, warm (optional, not well defined), and functional level reset. here we need fundamental that the actual PERST# signal is toggled on the slot.  
• Option 2: add hardware termination to the Lane 0 of SERDES1. I will ask our hardware team to send further details. 

Also please note that TDA4 support Distributed Refclk architecture, where the EP can use internal refclk on its own. The SERDES will recover the RX clock from the SERDES lanes. but understood you like to use common refclk architecture. 

3. Boot sequence if using TDA4 as RC

This is the default SDK support with the EVM. Suggested steps to test:

   1). verify the x4lane PCIe slot on the EVM works out of box with standard SDK, with a off-shelf NVME SSD installed on a NVME->PCIex4 adapter. 

   2). you can also verify the connect if you can connect the EVM to your EP hardware via a ribbon cable. 

The EVM uses an GPIO expander output to drive the PERST# signal, and use a separate CLKGEN device to supply the CLK on the slot. so you will need to the following changes for your own hardware:

    1). Change the PCIe reset signal to a standard GPIO in the device tree file

    2). Apply the patch to route refclk externally from TDA4, as explained in the FAQ Gary linked above. 

The RC driver in the SDK already took care of the reset/clk sequence. 

Regards

Jian

Yongliang, 

just checked the TRM, the SERDES1 PLL lock status register definition is not there. but you can use the same definition from the 4-L SERDES, please look under register name: 

PHY_PMA_CMN_CTRL2__PHY_PMA_CMN_CTRL1

regards

Jian

Hi jian,

  For option #2， could you help involve hardware team to update details?

Thanks.

I test three case about pcie's internal or external clock :

case 1: pcie1 in ep model whitout external clock

I change the device tree  set pcie1 in ep model

after compile and install,I test  the result is here:

in /sys/kernel/config/pci_ep/controllers,there is no pcie device

read register SERDES_RST(0x0501040c) is 0x92000000

read register PHY_PMA_CMN_CTRL2_PHY_PMA_CMN_CTRL1 is 0x00002435

case 2:pcie1 in ep model with external clock

I test  the result is here:

in /sys/kernel/config/pci_ep/controllers,there has pcie device

read register SERDES_RST(0x0501040c) is 0x92000000

read register PHY_PMA_CMN_CTRL2_PHY_PMA_CMN_CTRL1 is 0x00002435

case 3:pcie1 in rc mdel

I change the device tree  set pcie1 in rc model

read register SERDES_RST(0x0501040c) is 0x92000000

read register PHY_PMA_CMN_CTRL2_PHY_PMA_CMN_CTRL1 is 0x00002435

notes from 9/8:

potential solution for EP (TDA4) to use external refclk provided from Xavier.  both TDA4 and Xavier boot independently. therefore TDA4-EP will not have refclk by the time epf driver get loaded. 

Option 1: restart EPF driver after boot. SERDES not locked. Jian to confirm with driver team. (simplest)

Option 2: clean solution - Xavier to send a PERST per PCE standard. TDA4 to use the reset signal to trigger the loading of EPF, instead of driver get loaded at boot time. 

Option 3: use internal refclk then switch to external. - this is not going to work, as the glitch switching from internal to external may violate PCIE refclk jitter requirement. 

On high-speed design:

https://www.ti.com/lit/an/spracp4/spracp4.pdf?ts=1662616792357&ref_url=https%253A%252F%252Fwww.google.com%252F

Jian

Hi Jian,

     Any update from driver team about option 1?

Thanks.