Linux/AM5728: PCIe EP reference clock

yu chen

Expert 1455 points

Part Number: AM5728

Tool/software: Linux

Hi,

I am using the custom board of AM5728-EVM and ti-processor-sdk-linux-am57xx-evm-04.01.00.06.

we want to use pcie to communicate with windows pc.

Here is what we meet:

first,if we want to communicate with windows pc and the am5728-evm as EP mode,the am5728-evm must use the clock from the RC(windows PC) and use the synchronize mode.

synchronize mode change by CM_CLKMODE_APLL_PCIE,bit[7] 0x1: APLL reference input clock is from ACSPCIE.

we change the CTRL_CORE_SMA_SW_6.PCIE_TX_RX_CONTROL bit to change the ACSPCIE Ref. clock buffer mode as RX_mode.

But the windows do not recognize am5728-evm.

For confirm we change the CTRL_CORE_SMA_SW_6.PCIE_TX_RX_CONTROL at uboot is right,so we change it to TX_mode,and the CM_CLKMODE_APLL_PCIE,bit[7] 0x0: APLL reference input clock is from ADPLL.

ljcb_clkp and ljcb_clkn do not have 100Mhz clock.I use devmem2 tool to check the register value,the value has been changed.

Here is my questions:

(1) we can use two am5728-evm,one as RC and another as EP,it can work fine at asynchronous mode.so,I think the PCIE hardware design is fine.

(2)am5728-evm use pcie to communicate with windows pc,we must set ACSPCIE buffer as RX_mode,and change the APLL reference input clock is from ACSPCIE,am I right?

(3)I change the register 0x4a00341c and 0x4a00821c at uboot,we can see it is changed at filesystem,why the ljcb_clkp and ljcb_clkn don't have signal output at TX_MODE?

(4)if we can't change the register at uboot,can you tell me where can I change it?

(5)finally, our purpose is the am5728-evm(EP) use pcie to communicate with windows pc(RC), TI have some documents about this?

Any help will be appreaciate!

Thanks!

over 7 years ago

0 Biser Gatchev-XID over 7 years ago

TI__Guru**** 393215 points

The PCIe experts have been notified. They will respond here.

0 Rex Chang over 7 years ago in reply to Biser Gatchev-XID

TI__Guru 50170 points

Hi, CY,

I need to get some background info first. Is your board design based on AM5728-IDK or AM5728 GP EVM? In (1), when you refer to the AM5728 EVM, do you mean your custom board or TI EVM? That is, when 2 AM5728 EVM are connected, are both TI EVMs or both your boards? The registers change seems to be correct. I do know that I can't verify the RX mode on TI EVM without hardware modification. For Tx mode, if I remember correctly that on TI GP EVM, I need to remove C426 and C427, and the parallel 100 ohm resister between ljcb_clkn and ljcb_clkp. Both ljcb_clkn and ljcb_clkp should be independently terminated to GND with 50 ohms. The same should apply if it is based on AM5728 IDK but at C438 and C439.

Rex

0 Rex Chang over 7 years ago in reply to Rex Chang

TI__Guru 50170 points

Hi, CY,

There may be other issues when connecting to a windows PC. Please see it in the forum discussion, e2e.ti.com/.../630321

Rex

0 yu chen over 7 years ago in reply to Rex Chang

Expert 1455 points

Hi,Rex

first of all,thanks for your reply!
(1) we design based on AM5728 GP EVM.
(2) when I refer to the AM5728 EVM, I mean my board
(3) when 2 AM5728 EVM are connected,they are both our board.
(4) Thanks for your advise,my board's do not remove the "C426" and "C427" and the 100 ohm resister,I will remove it and check it again.

(5) something i want to confirm,at TI community, i found that someone has sloved this problem,but he used the SDK-3.00.00.04-linux,he changed the register at uboot successfuly.but the SDK-3.00.00.04-linux do not support EP mode.

I check the link,something i want to confirm,

a.remove the "C426" and "C427" and the 100 ohm resister will affect the RX_mode?

b."It seems to me that a Windows RC driver for AM57x is needed, but TI does not provide it." window RC driver,where can I find it?

c. what do you mean RX mode on TI EVM must modification the hardware? where should I modify?

d.EP side must power up and configure before RC side?so,we can't use the RC（windows-PC）pcie interface to provide power to the board,am I right for this?

0 Rex Chang over 7 years ago in reply to yu chen

TI__Guru 50170 points

Hi, CY,

Removing C426, C427 and 100 ohms resistor between clkp and clkn are for the Tx mode verification. It disconnects the ljcb_clkp/n from CDCM external clock. So, when ACSPCIE is programmed to be in Tx mode, the frequency can be measured at ljcb pins. To verify Rx mode on TI AM572x GP EVM, it requires other modification which we haven't done. The Rx mode was verified on different variant of the silicon platform. I don't have the info of mod needed for AM572x GP EVM, but for the variant, J6. platforms. I am not clear how it maps to AM572x GP EVM,

•PCIE Connector provides REFCLK to J6 (Vayu) EVM

–Depopulate capacitors C120/C125/C259/C262

–Populate 0-ohm resistors across C120/C259 and C125/C262 as shown in the diagram

–Depopulate resistors R806/R807

–Resistor R223

•Populate with 100ohm if PCIE Connector clock is LVDS

•Depopulate if PCIE Connector clock is HCSL

The J6 EVM schematics is in http://www.ti.com/tool/J6EVM5777

Could you provide the link to the thread which people had it done using 3.0.0.4? It shouldn't matter if it is RC or EP.

In the Windows environment, the PCIe cards are always plugged in before powering up the PC. I am not knowledgeable enough to know if there is hot plug case in Windows environment. About the windows driver, it will need a driver corresponding to the endpoint function. In Linux, we use pci-epf-test.c(ep side)/pci_endpoint_test(host side). So, if the pci-epf-test.c on the EP side is used, a windows PCI driver similar to pci_endpoint_test.c in linux is required.

Rex

0 yu chen over 7 years ago in reply to Rex Chang

Expert 1455 points

Hi,Rex

Thanks for your advises. but,we have verify the hardware,it don't work.

Because we use our boards,I want to exclude the hardware problems.the rx mode test has many uncertainties.

I want to focus on tx_mode to confirm my software is right.

I remove the "C426" and "C427" at our board,but we do not remove the 100 ohm resister.

we use oscilloscope to cat the signal at the resister.(if it is normal,we will cat 50Mhz signal,am i right?) but we do not cat any signal.

Then,we use RTOS-PCIE_idkAM572x_wSoCFile_armExampleProject.change it as tx_mode(HW_WR_FIELD32(SOC_SEC_EFUSE_REGISTERS_BASE + CSL_CONTROL_CORE_SEC_SMA_SW_6,CSL_CONTROL_CORE_SEC_SMA_SW_6_PCIE_TX_RX_CONTROL, 0x01U);

we use both arm or dsp example project run at our board.still can't cat any signal at ljcb_clkn and ljcb_clkp.
also,I use the ti-processor-sdk-linux-am57xx-evm-03.00.00.04, it is fail too.
e2e.ti.com/.../1750650

If you have some ideas,please let me know.

Thanks.

0 Rex Chang over 7 years ago in reply to yu chen

TI__Guru 50170 points

Hi, CY,

As my earlier post said "the parallel 100 ohm resister between ljcb_clkn and ljcb_clkp. Both ljcb_clkn and ljcb_clkp should be independently terminated to GND with 50 ohms".

Please give it a try.

Rex

0 yu chen over 7 years ago in reply to Rex Chang

Expert 1455 points

Hi,Rex

you are right!!!! I was wrong about the "100 ohm resister".

Now,I can cat 100Mhz at tx_mode.then I think i change the register actually.

here is my logs:

root@am57xx-evm:/sys/kernel/config/pci_ep# devmem2 0x4a003c14

/dev/mem opened.

Memory mapped at address 0xb6ff0000.

Read at address 0x4A003C14 (0xb6ff0c14): 0x00020000

root@am57xx-evm:/sys/kernel/config/pci_ep# devmem2 0x4a00821c

/dev/mem opened.

Memory mapped at address 0xb6f60000.

Read at address 0x4A00821C (0xb6f6021c): 0x00000181

root@am57xx-evm:/sys/kernel/config/pci_ep/

but when i put the board into PC.the PC don't recognize it either.

please let me know.

if I want the PC recognize out board.only I should do is change the 0x4a003c14 and 0x4a00821c?

any ideas do you have?

0 Rex Chang over 7 years ago in reply to yu chen

TI__Guru 50170 points

Hi, CY,

Glad to hear that you verified the Tx mode. For Rx mode, you need the hardware modification as in my earlier post for PCIe connector provides refclk to the am5x device. You may want to verify that you get the clock measurement at ljcb_clkp/ljcb_clkn.

Rex

0 yu chen over 7 years ago in reply to Rex Chang

Expert 1455 points

HI,Rex
Thanks for your help~
Yes,I have verified the hardware as your advise.
may be is because of the windows system?I want to try this at linux.
TI have any successful project about windows communication via pcie?

0 Rex Chang over 7 years ago in reply to yu chen

TI__Guru 50170 points

CY,

TI only tried between 2 EVMs, one as RC and the other as EP. We didn't set up with any PC, Windows or Linux.

Rex

0 yu chen over 7 years ago in reply to Rex Chang

Expert 1455 points

Hi,Rex

Sorry for the late reply...

Something i want to ask.

When I insert the board into PC,the LTSSM_EN bit is always 0,even though I use devmem2 tool to change it.

the datesheets have few information about the LTSSM_EN.

Can you explan something about this bit about force to 0? TI have some examples about "LTSSM_EN force to zero"?

And the PCIE PERST# pin,we don't set it at our dts file.Is it matter at this time?

There is a link about this solution,but I have change it as RX and ref clock as ACSPCIE buffer.

e2e.ti.com/.../2259892

0 Rex Chang over 7 years ago in reply to yu chen

TI__Guru 50170 points

Hi, CY,

I am not sure what you mean the "LTSSM_EN force to zero example". When LTSSM_EN is set to 0, it means to disable the PCIe link, is that what you want? When PCIe link is established, that bit will be set to 1. You can see it gets set in dra7xx_pcie_establish_link() in dwc/pci-dra7xx.c. So, when you read the bit being 0, it means the link training failed.

Rex

0 yu chen over 7 years ago in reply to Rex Chang

Expert 1455 points

Hi,Rex

Thanks for your help.Your advice help me a lot.

I still can't finish the issue.When I put the am5728 evm board in the PC PCI slot.the LTSSM_EN bit is 0.

I use devmem2 to set the LTSSM_EN bit,it will be cleared to 0 automatically.I can't find the reason.But sometimes the link is up at PCIE_idkAM572x_wSoCFile_armExampleProject.

If we want to communicate with the PC and am5728 evm board as EP,the PEREST signal is important at this situation?

If you have ideas about why the LTSSM_EN bit is always cleared,I will appreciate your help.

Anyway,I will close this thread lately,because the thread is about PCIe EP reference clock,I can confirm the hardware design is right now.

Thanks your help and sorry for my pool english.

0 Rex Chang over 7 years ago in reply to yu chen

TI__Guru 50170 points

CY,

I'll close this thread as you said that the reference clock issue is solved. Please submit a new thread for the new issue.

Rex

0 yu chen over 7 years ago in reply to Rex Chang

Expert 1455 points

Hi,Rex
Thanks for your help.
I can detect the board at PC now.

0 Rex Chang over 7 years ago in reply to yu chen

TI__Guru 50170 points

Hi, CY,

Do you mind sharing what you have changed to get it working? Thanks!

Rex

0 yu chen over 7 years ago in reply to Rex Chang

Expert 1455 points

Hi,Rex

Sorry,I am just lucky.I am not so clear.

Just like my earlier post,when the windows BIOS begin.the LTSSM_EN will be clear to 0,even thought i use devmem2 to set this bit.the LTSSM_EN is 0,the link is never up.

My PC PCIe slot do not support hot-plug,so I can't sure the am5728-evm(EP) will power up before the PC(RC) .

But I found something wired.when the PC is hung at "CMOS BIOS",the link is up,and the windows system will detect the am5728-evm.

I am not so familiar with PC,I don't know what is different.

BTW,I am no need to modify the hardware for detection.am5728-evm use the asynchronous clock.but I am not sure if it will affect the communication.

I can't communicate with the PC yet.I use windriver to create a universal PCI driver.but the windows has error report "The device can not find enough resources to use. (Code 12)".

I check the BAR size is not too large and the windows IRQ is not conflict.

somebody say is about the wrong EP configure at PC,I can't figure it out yet.If somebody know the difference,please send me some links about these.

Thanks!

0 Rex Chang over 7 years ago in reply to yu chen

TI__Guru 50170 points

Hi, CY,

Thanks for sharing your info.

I am not familiar with the issues interfacing with Windows PC, However, I think the host side windows PCI driver corresponding to the endpoint function is required. In linux we use the same pci-epf-test.c(EP side)/pci_endpoint_test(host side) for both AM57x. So if you use pci-epf-test.c on the EP side, a windows PCI driver (similar to pci_endpoint_test.c used in linux) is required.

I'll close this thread.

Rex

0 user3910344 over 7 years ago in reply to yu chen

Prodigy 130 points

Hi，CY
Do you solve the problem about the PC driver error ？ I meet the same problem. If you solve the problem，can you give me some advices？

0 yu chen over 7 years ago in reply to user3910344

Expert 1455 points

HI，

Yes, i fix the code 12 error.

different from dm8168/dm8148 pcie boot. am5728 finish the pcie init at kernel,which is too late.

If you use win7,the bios can't detect the pcie as ep,and bios will not give it resources, which calls code 12.

try win10,win10 will give it resources at os system.but I still meet code 10(the driver can not start).

I think it is because the driver can not enable the memory space enable bit,although the interrupt is wake up.

good luck.

Processors

Processors forum

Linux/AM5728: PCIe EP reference clock