• State TI Thinks Resolved
  • Locked Locked
  • Replies 14 replies
  • Answers 2 answers
  • Subscribers 101 subscribers
  • Views 1576 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM5716: PCIe access error.

Meysam Peykanu
Prodigy 110 points
Part Number: AM5716

Tool/software: Linux

We have a custom board with AM5716 connected to PEX9794 and A 10Gb Ethernet switch via the two PCIe interfaces. Sometime during the initialization, we get the error:

omap_l3_noc 44000000.ocp: L3 application error: target 5 mod:1 (unclearable)
omap_l3_noc 44000000.ocp: L3 debug error: target 5 mod:1 (unclearable)

In such cases the PEX9794 is not recognized on the PCIe bus but the other device is there. When probing the non-working case it can be seen that PCIe lane from PEX9794 is still active but the lane from the AM5716 is silent.

I have studied the related questions in the forum and it seems that there should be a power/reset problem but after checking all related signals, I have ran out of options. The current workaround is a complete reset. Is there a way to reset the PCIe bus of AM5716 and do a rescan inside Linux?

Here is the log from a working and non-working case.

Working:

[ 1.417909] pinctrl-single 4a003400.pinmux: 282 pins at pa fc003400 size 1128
[ 1.418038] pinctrl-single 4a002e8c.pinmux: 1 pins at pa fc002e8c size 4
[ 1.421029] dra7-pcie 51000000.pcie_rc: GPIO lookup for consumer (null)
[ 1.421041] dra7-pcie 51000000.pcie_rc: using device tree for GPIO lookup
[ 1.421053] of_get_named_gpiod_flags: can't parse 'gpios' property of node '/ocp/axi@0/pcie_rc@51000000[0]'
[ 1.421063] of_get_named_gpiod_flags: can't parse 'gpio' property of node '/ocp/axi@0/pcie_rc@51000000[0]'
[ 1.421072] dra7-pcie 51000000.pcie_rc: using lookup tables for GPIO lookup
[ 1.421082] dra7-pcie 51000000.pcie_rc: lookup for GPIO (null) failed
[ 1.421221] PCI host bridge /ocp/axi@0/pcie_rc@51000000 ranges:
[ 1.421232] No bus range found for /ocp/axi@0/pcie_rc@51000000, using [bus 00-ff]
[ 1.421270] IO 0x20003000..0x20012fff -> 0x00000000
[ 1.421291] MEM 0x20013000..0x2fffffff -> 0x20013000
[ 1.432153] dra7-pcie 51000000.pcie_rc: PCI host bridge to bus 0000:00
[ 1.432167] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 1.432178] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 1.432188] pci_bus 0000:00: root bus resource [mem 0x20013000-0x2fffffff]
[ 1.432223] pci 0000:00:00.0: [104c:8888] type 01 class 0x060400
[ 1.432266] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 1.432287] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x0000ffff]
[ 1.432357] pci 0000:00:00.0: supports D1
[ 1.432366] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[ 1.432605] PCI: bus0: Fast back to back transfers disabled
[ 1.432784] pci 0000:01:00.0: [11ab:c81d] type 00 class 0x020000
[ 1.432930] pci 0000:01:00.0: reg 0x10: [mem 0xd0000000-0xd00fffff 64bit pref]
[ 1.432987] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x03ffffff 64bit pref]
[ 1.433042] pci 0000:01:00.0: reg 0x20: [mem 0xf0000000-0xf07fffff 64bit pref]
[ 1.433280] pci 0000:01:00.0: supports D1 D2
[ 1.453136] PCI: bus1: Fast back to back transfers disabled
[ 1.453280] pci 0000:00:00.0: BAR 15: assigned [mem 0x22000000-0x27ffffff pref]
[ 1.453292] pci 0000:00:00.0: BAR 0: assigned [mem 0x20100000-0x201fffff]
[ 1.453305] pci 0000:00:00.0: BAR 1: assigned [mem 0x20020000-0x2002ffff]
[ 1.453322] pci 0000:01:00.0: BAR 2: assigned [mem 0x24000000-0x27ffffff 64bit pref]
[ 1.453370] pci 0000:01:00.0: BAR 4: assigned [mem 0x22000000-0x227fffff 64bit pref]
[ 1.453416] pci 0000:01:00.0: BAR 0: assigned [mem 0x22800000-0x228fffff 64bit pref]
[ 1.453461] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 1.453475] pci 0000:00:00.0: bridge window [mem 0x22000000-0x27ffffff pref]
[ 1.453698] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt
[ 1.453708] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[ 1.453720] pcie_pme 0000:00:00.0:pcie01: service driver pcie_pme loaded
[ 1.453843] aer 0000:00:00.0:pcie02: service driver aer loaded
[ 1.454274] dra7-pcie 51800000.pcie: GPIO lookup for consumer (null)
[ 1.454284] dra7-pcie 51800000.pcie: using device tree for GPIO lookup
[ 1.454295] of_get_named_gpiod_flags: can't parse 'gpios' property of node '/ocp/axi@1/pcie@51800000[0]'
[ 1.454305] of_get_named_gpiod_flags: can't parse 'gpio' property of node '/ocp/axi@1/pcie@51800000[0]'
[ 1.454314] dra7-pcie 51800000.pcie: using lookup tables for GPIO lookup
[ 1.454322] dra7-pcie 51800000.pcie: lookup for GPIO (null) failed
[ 1.454433] PCI host bridge /ocp/axi@1/pcie@51800000 ranges:
[ 1.454443] No bus range found for /ocp/axi@1/pcie@51800000, using [bus 00-ff]
[ 1.454475] IO 0x30003000..0x30012fff -> 0x00000000
[ 1.454495] MEM 0x30013000..0x3fffffff -> 0x30013000
[ 1.460279] dra7-pcie 51800000.pcie: PCI host bridge to bus 0001:00
[ 1.460292] pci_bus 0001:00: root bus resource [bus 00-ff]
[ 1.460304] pci_bus 0001:00: root bus resource [io 0x10000-0x1ffff] (bus address [0x0000-0xffff])
[ 1.460314] pci_bus 0001:00: root bus resource [mem 0x30013000-0x3fffffff]
[ 1.460348] pci 0001:00:00.0: [104c:8888] type 01 class 0x060400
[ 1.460388] pci 0001:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 1.460409] pci 0001:00:00.0: reg 0x14: [mem 0x00000000-0x0000ffff]
[ 1.460471] pci 0001:00:00.0: supports D1
[ 1.460481] pci 0001:00:00.0: PME# supported from D0 D1 D3hot
[ 1.460711] PCI: bus0: Fast back to back transfers disabled
[ 1.460892] pci 0001:01:00.0: [10b5:9749] type 01 class 0x060400
[ 1.461392] pci 0001:01:00.0: PME# supported from D0 D3hot D3cold
[ 1.473131] PCI: bus1: Fast back to back transfers disabled
[ 1.473149] pci 0001:01:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 1.473403] pci_bus 0001:02: busn_res: can not insert [bus 02-ff] under [bus 01] (conflicts with (null) [bus 01])
[ 1.473510] pci 0001:02:05.0: [10b5:9749] type 01 class 0x060400
[ 1.474038] pci 0001:02:05.0: PME# supported from D0 D3hot D3cold
[ 1.474472] pci 0001:02:09.0: [10b5:9749] type 01 class 0x060400
[ 1.474999] pci 0001:02:09.0: PME# supported from D0 D3hot D3cold
[ 1.475495] pci 0001:02:19.0: [10b5:9749] type 01 class 0x060400
[ 1.475919] pci 0001:02:19.0: PME# supported from D0 D3hot D3cold
[ 1.476309] PCI: bus2: Fast back to back transfers disabled
[ 1.476326] pci 0001:02:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 1.476360] pci 0001:02:09.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 1.476394] pci 0001:02:19.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 1.476655] PCI: bus3: Fast back to back transfers enabled
[ 1.476669] pci_bus 0001:03: busn_res: [bus 03-ff] end is updated to 03
[ 1.476686] pci_bus 0001:03: [bus 03] partially hidden behind bridge 0001:01 [bus 01]
[ 1.476930] PCI: bus4: Fast back to back transfers enabled
[ 1.476941] pci_bus 0001:04: busn_res: [bus 04-ff] end is updated to 04
[ 1.476957] pci_bus 0001:04: [bus 04] partially hidden behind bridge 0001:01 [bus 01]
[ 1.477257] pci 0001:05:00.0: [10b5:1009] type 00 class 0x088000
[ 1.477383] pci 0001:05:00.0: reg 0x10: [mem 0x00000000-0x007fffff]
[ 1.478405] PCI: bus5: Fast back to back transfers disabled
[ 1.478418] pci_bus 0001:05: busn_res: [bus 05-ff] end is updated to 08
[ 1.478435] pci_bus 0001:05: [bus 05-08] partially hidden behind bridge 0001:01 [bus 01]
[ 1.478450] pci_bus 0001:02: busn_res: [bus 02-ff] end is updated to 08
[ 1.478461] pci_bus 0001:02: busn_res: can not insert [bus 02-08] under [bus 01] (conflicts with (null) [bus 01])
[ 1.478477] pci_bus 0001:02: [bus 02-08] partially hidden behind bridge 0001:01 [bus 01]
[ 1.478491] pci 0001:00:00.0: bridge has subordinate 01 but max busn 08
[ 1.479028] pci 0001:00:00.0: BAR 14: assigned [mem 0x30800000-0x30ffffff]
[ 1.479040] pci 0001:00:00.0: BAR 0: assigned [mem 0x30100000-0x301fffff]
[ 1.479053] pci 0001:00:00.0: BAR 1: assigned [mem 0x30020000-0x3002ffff]
[ 1.479068] pci 0001:01:00.0: BAR 14: assigned [mem 0x30800000-0x30ffffff]
[ 1.479080] pci 0001:02:19.0: BAR 14: assigned [mem 0x30800000-0x30ffffff]
[ 1.479089] pci 0001:02:05.0: PCI bridge to [bus 03]
[ 1.479142] pci 0001:02:09.0: PCI bridge to [bus 04]
[ 1.479198] pci 0001:05:00.0: BAR 0: assigned [mem 0x30800000-0x30ffffff]
[ 1.479218] pci 0001:02:19.0: PCI bridge to [bus 05-08]
[ 1.479241] pci 0001:02:19.0: bridge window [mem 0x30800000-0x30ffffff]
[ 1.479279] pci 0001:01:00.0: PCI bridge to [bus 02-08]
[ 1.479302] pci 0001:01:00.0: bridge window [mem 0x30800000-0x30ffffff]
[ 1.479338] pci 0001:00:00.0: PCI bridge to [bus 01]
[ 1.479349] pci 0001:00:00.0: bridge window [mem 0x30800000-0x30ffffff]
[ 1.479560] pcieport 0001:00:00.0: Signaling PME through PCIe PME interrupt
[ 1.479569] pci 0001:01:00.0: Signaling PME through PCIe PME interrupt
[ 1.479577] pci 0001:02:05.0: Signaling PME through PCIe PME interrupt
[ 1.479584] pci 0001:02:09.0: Signaling PME through PCIe PME interrupt
[ 1.479591] pci 0001:02:19.0: Signaling PME through PCIe PME interrupt
[ 1.479598] pci 0001:05:00.0: Signaling PME through PCIe PME interrupt
[ 1.479608] pcie_pme 0001:00:00.0:pcie01: service driver pcie_pme loaded
[ 1.479799] aer 0001:00:00.0:pcie02: service driver aer loaded
[ 1.479890] pcieport 0001:01:00.0: enabling device (0140 -> 0142)
[ 1.481866] pcieport 0001:02:19.0: enabling device (0140 -> 0142)

Non-working:

[ 1.417543] pinctrl-single 4a003400.pinmux: 282 pins at pa fc003400 size 1128
[ 1.417674] pinctrl-single 4a002e8c.pinmux: 1 pins at pa fc002e8c size 4
[ 1.420734] dra7-pcie 51000000.pcie_rc: GPIO lookup for consumer (null)
[ 1.420746] dra7-pcie 51000000.pcie_rc: using device tree for GPIO lookup
[ 1.420758] of_get_named_gpiod_flags: can't parse 'gpios' property of node '/ocp/axi@0/pcie_rc@51000000[0]'
[ 1.420768] of_get_named_gpiod_flags: can't parse 'gpio' property of node '/ocp/axi@0/pcie_rc@51000000[0]'
[ 1.420777] dra7-pcie 51000000.pcie_rc: using lookup tables for GPIO lookup
[ 1.420787] dra7-pcie 51000000.pcie_rc: lookup for GPIO (null) failed
[ 1.420924] PCI host bridge /ocp/axi@0/pcie_rc@51000000 ranges:
[ 1.420936] No bus range found for /ocp/axi@0/pcie_rc@51000000, using [bus 00-ff]
[ 1.420974] IO 0x20003000..0x20012fff -> 0x00000000
[ 1.420995] MEM 0x20013000..0x2fffffff -> 0x20013000
[ 1.432232] dra7-pcie 51000000.pcie_rc: PCI host bridge to bus 0000:00
[ 1.432247] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 1.432258] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 1.432267] pci_bus 0000:00: root bus resource [mem 0x20013000-0x2fffffff]
[ 1.432302] pci 0000:00:00.0: [104c:8888] type 01 class 0x060400
[ 1.432344] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 1.432366] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x0000ffff]
[ 1.432434] pci 0000:00:00.0: supports D1
[ 1.432444] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[ 1.432683] PCI: bus0: Fast back to back transfers disabled
[ 1.432862] pci 0000:01:00.0: [11ab:c81d] type 00 class 0x020000
[ 1.433005] pci 0000:01:00.0: reg 0x10: [mem 0xd0000000-0xd00fffff 64bit pref]
[ 1.433116] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x03ffffff 64bit pref]
[ 1.433173] pci 0000:01:00.0: reg 0x20: [mem 0xf0000000-0xf07fffff 64bit pref]
[ 1.433362] pci 0000:01:00.0: supports D1 D2
[ 1.453129] PCI: bus1: Fast back to back transfers disabled
[ 1.453273] pci 0000:00:00.0: BAR 15: assigned [mem 0x22000000-0x27ffffff pref]
[ 1.453285] pci 0000:00:00.0: BAR 0: assigned [mem 0x20100000-0x201fffff]
[ 1.453299] pci 0000:00:00.0: BAR 1: assigned [mem 0x20020000-0x2002ffff]
[ 1.453316] pci 0000:01:00.0: BAR 2: assigned [mem 0x24000000-0x27ffffff 64bit pref]
[ 1.453364] pci 0000:01:00.0: BAR 4: assigned [mem 0x22000000-0x227fffff 64bit pref]
[ 1.453410] pci 0000:01:00.0: BAR 0: assigned [mem 0x22800000-0x228fffff 64bit pref]
[ 1.453455] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 1.453469] pci 0000:00:00.0: bridge window [mem 0x22000000-0x27ffffff pref]
[ 1.453693] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt
[ 1.453703] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[ 1.453714] pcie_pme 0000:00:00.0:pcie01: service driver pcie_pme loaded
[ 1.453837] aer 0000:00:00.0:pcie02: service driver aer loaded
[ 1.454268] dra7-pcie 51800000.pcie: GPIO lookup for consumer (null)
[ 1.454279] dra7-pcie 51800000.pcie: using device tree for GPIO lookup
[ 1.454289] of_get_named_gpiod_flags: can't parse 'gpios' property of node '/ocp/axi@1/pcie@51800000[0]'
[ 1.454298] of_get_named_gpiod_flags: can't parse 'gpio' property of node '/ocp/axi@1/pcie@51800000[0]'
[ 1.454306] dra7-pcie 51800000.pcie: using lookup tables for GPIO lookup
[ 1.454315] dra7-pcie 51800000.pcie: lookup for GPIO (null) failed
[ 1.454427] PCI host bridge /ocp/axi@1/pcie@51800000 ranges:
[ 1.454438] No bus range found for /ocp/axi@1/pcie@51800000, using [bus 00-ff]
[ 1.454470] IO 0x30003000..0x30012fff -> 0x00000000
[ 1.454491] MEM 0x30013000..0x3fffffff -> 0x30013000
[ 1.460807] dra7-pcie 51800000.pcie: PCI host bridge to bus 0001:00
[ 1.460820] pci_bus 0001:00: root bus resource [bus 00-ff]
[ 1.460832] pci_bus 0001:00: root bus resource [io 0x10000-0x1ffff] (bus address [0x0000-0xffff])
[ 1.460842] pci_bus 0001:00: root bus resource [mem 0x30013000-0x3fffffff]
[ 1.460877] pci 0001:00:00.0: [104c:8888] type 01 class 0x060400
[ 1.460920] pci 0001:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 1.460941] pci 0001:00:00.0: reg 0x14: [mem 0x00000000-0x0000ffff]
[ 1.461001] pci 0001:00:00.0: supports D1
[ 1.461010] pci 0001:00:00.0: PME# supported from D0 D1 D3hot
[ 1.461242] PCI: bus0: Fast back to back transfers disabled
[ 1.485829] omap_l3_noc 44000000.ocp: L3 application error: target 5 mod:1 (unclearable)
[ 1.485847] omap_l3_noc 44000000.ocp: L3 debug error: target 5 mod:1 (unclearable)
[ 1.485859] PCI: bus1: Fast back to back transfers enabled
[ 1.485974] PCI: bus2: Fast back to back transfers enabled
[ 1.485987] pci_bus 0001:02: busn_res: [bus 02-ff] end is updated to 02
[ 1.486129] pci 0001:00:00.0: BAR 0: assigned [mem 0x30100000-0x301fffff]
[ 1.486144] pci 0001:00:00.0: BAR 1: assigned [mem 0x30020000-0x3002ffff]
[ 1.486156] pci 0001:00:00.0: PCI bridge to [bus 02]
[ 1.486215] pcieport 0001:00:00.0: enabling device (0000 -> 0002)
[ 1.486392] pcieport 0001:00:00.0: Signaling PME through PCIe PME interrupt
[ 1.486405] pcie_pme 0001:00:00.0:pcie01: service driver pcie_pme loaded
[ 1.486521] aer 0001:00:00.0:pcie02: service driver aer loaded

Thanks,

Meysam

  • 0
    TI__Guru 50170 points
    Hi, Meysam,

    Please take a look at the discussion on this thread, e2e.ti.com/.../701888, especially DK's comment. As he said, majority issues can be traced to layout issues.

    To rescan, please see this thread for more info, e2e.ti.com/.../745415

    Rex
  • 0 in reply to Rex Chang
    Prodigy 110 points

    Hi Rex,

    I just probed the received PCIe signals near Sitara and PEX side with TriMode probes and our high-end oscilloscope. You can see the eye diagrams here. As can be seen for the working condition, there are good eyes and the link is established in Gen 2. So it does not seem to be a signal integrity problem.

    PEX Rx: 

    AM5716 Rx:

    For the non-working condition the PEX output is Gen 1 and there is no signal received by the PEX from Sitara.

    AM5716 Rx:

    Using the following workaround solution did not help as well and the Sitara PCIe output is still off. ()

    echo 1 > "/sys/bus/pci/devices/0000:00:00.0/remove"

    echo 1 > /sys/bus/pci/rescan

    The question is, will the above commands actually do a "PCIe Subsystem Warm Reset" as mentioned in "24.9.4.4.2.1.2 PCIe Subsystem Warm Main Reset Sources" of the TRM?

    Do you have any suggestions regarding the source of the problem?

    Thanks,

    Meysam

  • 0 in reply to Meysam Peykanu
    TI__Mastermind 27720 points
    Hi Meysam,
    What version of the TI SDK are you using?
    Please report the LTSSM state (LTSSM_STATE) and status (LTSSM_EN) of a failed link-up condition.
  • 0 in reply to -DK-
    Prodigy 110 points

    The kernel we are using is "PROCESSOR-SDK-LINUX-AM57X 03.02.00.05" and we are not planning to change to a newer version anytime soon as major changes are needed. We have included the value print of the PCIECTRL_TI_CONF_DEVICE_CMD register in the boot log and it looks like this:

    non-working:
    [ 1.635807] dra7-pcie 51800000.pcie: CFG 'Bus Master Enable' change
    [ 1.635817] dra7-pcie 51800000.pcie: CFG 'Memory Space Enable' change
    [ 1.635827] dra7-pcie 51800000.pcie: PCIECTRL_DRA7XX_CONF_DEVICE_CMD = 0x0
    [ 1.635839] dra7-pcie 51800000.pcie: dra7xx_pcie_start_link, reg=0x1
    [ 1.636971] dra7-pcie 51800000.pcie: Link-up state change
    [ 1.636982] dra7-pcie 51800000.pcie: PCIECTRL_DRA7XX_CONF_DEVICE_CMD = 0x39
    [...]
    [ 1.662173] omap_l3_noc 44000000.ocp: L3 application error: target 5 mod:1 (unclearable)
    [ 1.662187] dra7-pcie 51800000.pcie: Link Request Reset
    [ 1.662195] dra7-pcie 51800000.pcie: Link-up state change
    [ 1.662203] dra7-pcie 51800000.pcie: CFG 'Bus Master Enable' change
    [ 1.662210] dra7-pcie 51800000.pcie: CFG 'Memory Space Enable' change
    [ 1.662219] dra7-pcie 51800000.pcie: PCIECTRL_DRA7XX_CONF_DEVICE_CMD = 0x0
    [ 1.662232] omap_l3_noc 44000000.ocp: L3 debug error: target 5 mod:1 (unclearable)

    After the boot process PCIECTRL_TI_CONF_DEVICE_CMD value for the working PCIe port is 0x45 and for the non-working port 0x0 which means LTSSM is not enabled anymore.

    So it seems that during the boot LTSSM enters RCVRY_SPEED(0x39 -->0x0E for LTSSM_STATE) and after some error resets with no retry for re-initialization. right? any reason or workaround suggestion?

    Thanks,

    Meysam

  • 0 in reply to Meysam Peykanu
    TI__Mastermind 27720 points
    Hello Meysam,
    I'll need our SW guy to answer the LTSSM retry/reset question as this behavior would be controlled by the driver. He is out for the Thanksgiving holiday so I wouldn't expect an answer until 12/3.

    I'll also suggest that the first thing he will ask you to do is to try the same test using the current SDK. SDK 3 is very old and there have been significant PCIe updates since it was released.
  • 0 in reply to -DK-
    Prodigy 110 points

    Hi,

    We tried the new SDK and the problem is still there.

    [    2.753899] dra7-pcie 51800000.pcie: phy link never came up

    and after booting is complete:

    devmem2 0x51802104
    /dev/mem opened.
    Memory mapped at address 0xb6f3a000.
    Value at address 0x51802104 (0xb6f3a104): 0x0

    So it is like before.

    Best regards,

    Meysam

  • 0 in reply to Meysam Peykanu
    TI__Mastermind 27720 points
    OK, thanks. Could you please post the PCIe portion of your schematic?
  • 0 in reply to -DK-
    TI__Mastermind 27720 points
    Also please post the trace-lengths for the PCIe buses.
  • 0 in reply to -DK-
    TI__Guru 50170 points
    Meysam,

    It seems that the PCIe is able to send and receive data, but can't establish a reliable speed. Hence, it enters the recovery state.
    As in the thread url I pointed out in the beginning, the pcie phy link never came up message may have something to do with not having good power source.

    Rex
  • 0 in reply to -DK-
    Prodigy 110 points

    Hi,

    Here is the PCIe schematic portion. A short description is that we have a PCIe multiplexer (PI3PCIE3212ZBE) in the middle which connects Sitara and PEX by default. The 3.3V power (like all other power rails) comes long before the PEX is out of reset. The clock is coming from the same PCIe clock generator for both devices. It is important to notice that the exact same circuit (same PCIe MUX, same clock source...) is used to manage a 10Gb Ethernet switch via the other PCIe port (port 0) which is working fine all the time.

    The trace length is almost 27 cm and 23 cm for Sitara RX and TX (including both sides of PCIe MUX) respectively.

    I still want to know how it is possible to restart the PCIe phy inside Linux. This might be good workaround for us. FYI removing the route complex and rescan didn't help although PEX is still active on the PCIe lane. 

    One more Question. Where should be modified to limit the PCIe speed to Gen 1?

    Thanks,

    Meysam

  • 0 in reply to Meysam Peykanu
    TI__Mastermind 27720 points
    Thanks Meysam.

    What is the PCIe tracelength from SoC to PCIe MUX?

    I've asked Rex to chime in for your "limiting PCIe speed" question.
  • 0 in reply to -DK-
    Prodigy 110 points

    Hi,

    20 cm and 19 cm for RX and TX lanes respectively.

    Thanks,

    Meysam

  • 0 in reply to Meysam Peykanu
    TI__Guru 50170 points
    Hi, Meysam,

    Please see linux Documentation in Documentation/devicetree/bindings/pci/pci.txt.

    Rex
  • 0 in reply to Meysam Peykanu
    TI__Mastermind 27720 points
    Hi Meysam,
    Your trace lengths significantly exceed the maximums permitted for this device. I think it likely that the SoC PCIe SerDes is having a hard time linking due to the resulting insertion losses.

    Please refer to SPRAAR7 for more detail on high-speed layout guidelines and requirements.