AM5728: PCIe force GEN1 operation

user4759774

Part Number: AM5728

Hi all,

Is there a way to force PCIe_SS{1,2} controller to operate with GEN1 (2.5Gb) speed?

I only have found MAX_LINK_SPEEDS [3:0] RW equal to 0x2 -> GEN2. ( PCIECTRL_EP_DBICS_LNK_CAP - 0x5180 007C register at AM5728 - SPRUHZ6G).

Setting it to 0x1 does not bring any change, since I do read 0x2 afterwards.

I need to force GEN1 operational speed on this port because of PCIe switch problem (Pericom) connected to it. The switch sometimes is not correctly recognized - LTSSM fails for this port.

Best regards,

Lukasz

over 7 years ago

0 lding over 7 years ago

TI__Guru* 95265 points

Hi,

Please refer to PCIE RTOS example code:

pcieRet_e pcieSetGen2(Pcie_Handle handle)

{

pcieRet_e retVal;

pcieRegisters_t regs;

pcieLinkCapReg_t linkCap;

pcieGen2Reg_t gen2;

uint8_t targetGen, dirSpd;

#ifdef GEN2

targetGen = 2;

dirSpd = 1;

#else

targetGen = 1;

dirSpd = 0;

#endif

.....

You need to set PCIECTRL_PL_WIDTH_SPEED_CTL bit 17 CFG_DIRECTED_SPEED_CHANGE to 0 as well.

Regards, Eric

0 user4759774 over 7 years ago in reply to lding

Intellectual 745 points

Hi Eric,

Do I need to do any special operations (like LINK_DIS, LINK_RETRAIN, etc) to take this change effect.

I've changed it just before LTSSM_EN = 1 and lspci -vvv shows
LnkSta: Speed 5GT/s

Best regards,
Łukasz

0 lding over 7 years ago in reply to user4759774

TI__Guru* 95265 points

Hi,

Do you mean you first have a link established at GEN2 speed then you want to down train it to GEN1 speed? Or, you want to bring the link at GEN1 speed at the beginning? I thought your usage case is the latter one. Please confirm?

In our RTOS example, we tested GEN1 only, then GEN2, either with PCIESS1 x 1 or x2 using our EVM. We didn't see any problem the GEN1/GEN2 speed we got vs setting. Even the LINK_CAP set to 0x2 (you mentioned 0x5180_007c, you use PCIESS2?), it doesn't matter. The link speed is controlled by dirSpd mentioned previously.

It is a surprise that you still get GEN2 with the setting. Please explain your configuration, PCIESS1 x1 or x2? Are you PCIE EP or RC? connection topology? what device runs Linux OS (you typed lspci)?

Regards, Eric

0 user4759774 over 7 years ago in reply to lding

Intellectual 745 points

Hi Eric,

I would like to establish GEN1 speed from the very beginning - form the first run of LTSSM (this is the one way to fix Pericom PCIe switch discovery issue).

I use PCIe_SS2 x1. It works as RC (standard) and I do have direct connection to PCIe switch. PCIe clock is provided by buffer. I do run Linux.

Working output for lspci:

0001:00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01)
0001:01:00.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)

Broken:

0001:00:00.0 Non-VGA unclassified device: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01)

Best regards,

Lukasz

0 lding over 7 years ago in reply to user4759774

TI__Guru* 95265 points

Lukasz,

Thanks for clarification! Due to EVM HW limitation, we don't have any test with PCIESS2x1. The PCIESS1 and PCIESS2 are identical controller on AM5728, the setting how to limit to GEN1 is the same, this is from PCIE Specification.

We tested using dirSpd field to limit speed to GEN1 on several TI devices: keystone I, keystone II, AM571x/2x.

Can you dump the working and failure case registers (JTAG or devmem2):

0x51802104
0x51801080
0x5180080c
0x5180007c

Regards, Eric

0 lding over 7 years ago in reply to lding

TI__Guru* 95265 points

Also,

I just tested a case: where RC (AM5728, setting of 0x5100007c BIT 3:0 with GEN1, 0x5100080c bit 17 = 0 to disable dirspd) ----- EP (AM5728, setting of 0x5100007c BIT 3:0 with GEN2, 0x5100080c bit 17 = 1 to enable dirspd), I only got GEN1 speed when link came up.

Regards, Eric

0 Robert Finger over 7 years ago in reply to lding

TI__Genius 16602 points

Hi Eric,

Setting PCIe to GEN1 would only be a workaround. Maybe the root cause could be fixed as well. As mentioned, the Pericom switch is sometimes detected, sometimes not. Attached Linux logs.

In the error case the Pericom switch is detected as Non-VGA device. Shouldn't this always show up as Texas Instruments device?

„lspci“ with non detected Pericom switch:

root@beaglebone:~# lspci
0000:00:00.0 PCI bridge: Texas Instruments Device 8888 (rev 01)
0000:01:00.0 Bridge: xxx xxxx xxxxx Multifunction IP core
0001:00:00.0 PCI bridge: Texas Instruments Device 8888 (rev 01)
0001:01:00.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:02:01.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:02:02.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:02:03.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:05:00.0 PCI bridge: Tundra Semiconductor Corp. Device 8114 (rev 03)
root@beaglebone:~#

„lspci“ if switch is not detected:

root@beaglebone:~# lspci
0000:00:00.0 PCI bridge: Texas Instruments Device 8888 (rev 01)
0000:01:00.0 Bridge: xxx xxxx xxxxx Multifunction IP core
0001:00:00.0 Non-VGA unclassified device: Texas Instruments Device 8888 (rev 01)
root@beaglebone:~#

„lspci“ with removed connection to Pericom switch (serial caps removed):

switch.zip

Thanks,
Robert

0 lding over 7 years ago in reply to Robert Finger

TI__Guru* 95265 points

Hi,

Thanks for the lspci/linux boot log for both cases. It showed working case the Link speed is GEN2 from "80: 48 00 12 f0 00 00 00 00 c0 03 40 00 08 00 00 00" also the LINK_CAP is GEN2 as well from "70: 10 00 42 00 01 80 00 00 1f 28 00 00 22 3c 73 00".

Due to you have some enumeration failure using GEN2, two paths:

1. Found the reason why GEN2 failed: Do you use the AM5728 in normal room tempeature and had this failure? Does the Pericom switch shows robust in GEN2 operation with other PCIE root complexs and no issue found? The connection between AM5728 and Pericom switch is secure and signal quality is good (any measurement for this)?

2. Limit speed to GEN1 as a workaround: this need to change 0x5180007C bit 3:0 MAX_LINK_SPEED field and 0x5180080C bit 17 to 0 on AM5728 side. I think you need to rebuild Linux kernel to do this (I am on bare metal RTOS side, not Linux). You mentioned that you changed the registers, you still got GEN2. I had a test yesterday showed I only got GEN1 if I used those changes I suggested. So I want you show me the dump that even you set RC side to GEN1 correctly, you still get GEN2. I don't understand why that doesn't work to limit to GEN1.

Regards, Eric

0 user4759774 over 7 years ago in reply to lding

Intellectual 745 points

Hi Eric, Robert,

lding said:

2. Limit speed to GEN1 as a workaround: this need to change 0x5180007C bit 3:0 MAX_LINK_SPEED field and 0x5180080C bit 17 to 0 on AM5728 side. I think you need to rebuild Linux kernel to do this (I am on bare metal RTOS side, not Linux). You mentioned that you changed the registers, you still got GEN2. I had a test yesterday showed I only got GEN1 if I used those changes I suggested. So I want you show me the dump that even you set RC side to GEN1 correctly, you still get GEN2. I don't understand why that doesn't work to limit to GEN1.

Regards, Eric

Please find attached output:

NOT changed (I've changed both controllers - but we would need only port1 to change):

root@beaglebone:~# dmesg|grep pcie_port
[ 0.405499] dra7xx_add_pcie_port : dev: 51000000.pcie: 0x5180 007c: 0x733c22 [B]
[ 0.405508] dra7xx_add_pcie_port : dev: 51000000.pcie: 0x5180 007c: 0x733c22 [A]
[ 0.405515] dra7xx_add_pcie_port : dev: 51000000.pcie: 0x5180 080c: 0x2020f [B]
[ 0.405521] dra7xx_add_pcie_port : dev: 51000000.pcie: 0x5180 080c: 0x2020f [A]
[ 0.520268] dra7xx_add_pcie_port : dev: 51800000.pcie: 0x5180 007c: 0x733c22 [B]
[ 0.520275] dra7xx_add_pcie_port : dev: 51800000.pcie: 0x5180 007c: 0x733c22 [A]
[ 0.520281] dra7xx_add_pcie_port : dev: 51800000.pcie: 0x5180 080c: 0x2020f [B]
[ 0.520287] dra7xx_add_pcie_port : dev: 51800000.pcie: 0x5180 080c: 0x2020f [A]

root@beaglebone:~# lspci
0000:00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01)
0000:01:00.0 Bridge: YYYYYYYYYYYYYYYYYYYYYYY Multifunction IP core
0001:00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01)
0001:01:00.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:02:01.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:02:02.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:02:03.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:05:00.0 PCI bridge: Tundra Semiconductor Corp. Device 8114 (rev 03)

root@beaglebone:~# lspci -s 0001:00:00.0 -vv | grep LnkCap
LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us

root@beaglebone:~# lspci -s 0001:00:00.0 -vv | grep LnkSta
LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt+
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-

root@beaglebone:~# lspci -s 0001:01:00.0 -vv | grep LnkSta
LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-

root@beaglebone:~# lspci -s 0001:01:00.0 -vv | grep LnkCap
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported, Exit Latency L0s <512ns, L1 <1us

The above looks as expected - Speed: 5GT/s

CHANGED:

root@beaglebone:~# lspci
0000:00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01)
0000:01:00.0 Bridge: YYYYYYYYYYYYYYYYYYYYYYYYY Multifunction IP core
0001:00:00.0 PCI bridge: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01)
0001:01:00.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:02:01.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:02:02.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:02:03.0 PCI bridge: Pericom Semiconductor Device 2404 (rev 05)
0001:05:00.0 PCI bridge: Tundra Semiconductor Corp. Device 8114 (rev 03)
root@beaglebone:~# dmesg|grep pcie_port
[ 0.405623] dra7xx_add_pcie_port : dev: 51000000.pcie: 0x5180 007c: 0x733c22 [B]
[ 0.405631] dra7xx_add_pcie_port : dev: 51000000.pcie: 0x5180 007c: 0x733c21 [A]
[ 0.405638] dra7xx_add_pcie_port : dev: 51000000.pcie: 0x5180 080c: 0x2020f [B]
[ 0.405645] dra7xx_add_pcie_port : dev: 51000000.pcie: 0x5180 080c: 0x20f [A]
[ 0.519795] dra7xx_add_pcie_port : dev: 51800000.pcie: 0x5180 007c: 0x733c22 [B]
[ 0.519802] dra7xx_add_pcie_port : dev: 51800000.pcie: 0x5180 007c: 0x733c21 [A]
[ 0.519809] dra7xx_add_pcie_port : dev: 51800000.pcie: 0x5180 080c: 0x2020f [B]
[ 0.519815] dra7xx_add_pcie_port : dev: 51800000.pcie: 0x5180 080c: 0x20f [A]

root@beaglebone:~# lspci -s 0001:00:00.0 -vv | grep LnkCap
LnkCap: Port #0, Speed 2.5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us

root@beaglebone:~# lspci -s 0001:00:00.0 -vv | grep LnkSta
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt+
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-

root@beaglebone:~# lspci -s 0001:01:00.0 -vv | grep LnkCap
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported, Exit Latency L0s <512ns, L1 <1us

root@beaglebone:~# lspci -s 0001:01:00.0 -vv | grep LnkSta
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-

Here output also as expected - the Pericom PCIe bridge can have 5GT/s speed setup.

I'm now testing the communication with GEN1. Looks promissing - 30x power cycles and no failure. I will let you know if we fail.

Best regards,

Łukasz

0 user4759774 over 7 years ago in reply to user4759774

Intellectual 745 points

One more remark:

When I do NOT change those registers - from time to time I do have the following error:

Sometimes Pericom is not detected when GEN2 set:
================================================

cm12#0>mdahb 0x5180007c
AHB/AXI 00_5180007c : 00733c22

cm12#0>mdahb 0x5180080c
AHB/AXI 00_5180080c : 0002020f

root@beaglebone:~# lspci -s 0001:00:00.0 -vv
0001:00:00.0 Non-VGA unclassified device: Texas Instruments Multicore DSP+ARM KeyStone II SOC (rev 01)
!!! Invalid class 0000 for header type 01
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 0
Region 0: Memory at <unassigned> (32-bit, prefetchable) [disabled]
Region 1: Memory at <unassigned> (32-bit, prefetchable) [disabled]
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 00000000-00000fff
Memory behind bridge: 00000000-000fffff
Prefetchable memory behind bridge: 00000000-000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0
ExtTag- RBE+
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-

However, up till now with setting GEN1 speed I was not able to observe broken lspci output for Pericom PCIe bridge.

BR,
Łukasz

0 lding over 7 years ago in reply to user4759774

TI__Guru* 95265 points

Lukasz,

To re-cap, setting those registers does limit the speed to GEN1, and it does achieve a stable PCIE enumeration with your limited (+30 times) reboot test. So you can use the workaround for now.

As for the root cause why GEN2 has enumearion failure from time to time, please open a new ticket when you re-visit there.

Regards, Eric

0 Robert Finger over 7 years ago in reply to lding

TI__Genius 16602 points

Hi Eric,

I've created a new thread to discuss the GEN2 detection issue:
https://e2e.ti.com/support/arm/sitara_arm/f/791/t/567936

Thanks,
Robert

Processors

Processors forum

AM5728: PCIe force GEN1 operation