• State Resolved
  • Locked Locked
  • Replies 6 replies
  • Subscribers 101 subscribers
  • Views 4015 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM5728: Need to configure PCIe RC to expect an attached PCIe switch

Chris Welch66
Expert 1790 points
Part Number: AM5728

Tool/software: Linux

We have an AM5728 product based on the TI EVM.  We have two PEX 8606 PCIe switches attached to the PCI RC. 

We have split the controller into two separate bridges as discussed in https://e2e.ti.com/support/embedded/linux/f/354/t/595948. 

[ 1.847038] PCI host bridge /ocp/axi@0/pcie_rc@51000000 ranges:
[ 1.847050] No bus range found for /ocp/axi@0/pcie_rc@51000000, using [bus
00-ff]
[ 1.847084] IO 0x20003000..0x20012fff -> 0x00000000
[ 1.847104] MEM 0x20013000..0x2fffffff -> 0x20013000
[ 1.849650] dra7-pcie 51000000.pcie_rc: PCI host bridge to bus 0000:00
[ 1.849665] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 1.849676] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 1.849687] pci_bus 0000:00: root bus resource [mem 0x20013000-0x2fffffff]
[ 1.849721] pci 0000:00:00.0: [104c:8888] type 01 class 0x060400
[ 1.849764] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 1.849786] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x0000ffff]
[ 1.849853] pci 0000:00:00.0: supports D1
[ 1.849863] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[ 1.850116] PCI: bus0: Fast back to back transfers disabled
[ 1.850281] pci 0000:01:00.0: [10b5:8606] type 00 class 0x060400
[ 1.850311] pci 0000:01:00.0: ignoring class 0x060400 (doesn't match header t
ype 00)
[ 1.850560] PCI: bus1: Fast back to back transfers disabled
[ 1.850640] pci 0000:00:00.0: BAR 0: assigned [mem 0x20100000-0x201fffff]
[ 1.850655] pci 0000:00:00.0: BAR 1: assigned [mem 0x20020000-0x2002ffff]
[ 1.850668] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 1.850914] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt
[ 1.850924] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[ 1.850936] pcie_pme 0000:00:00.0:pcie01: service driver pcie_pme loaded
[ 1.851073] aer 0000:00:00.0:pcie02: service driver aer loaded

debian@arm:/boot$ sudo lspci
[sudo] password for debian:
0000:00:00.0 PCI bridge: Texas Instruments Device 8888 (rev 01)
0000:01:00.0 Non-VGA unclassified device: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
0001:00:00.0 PCI bridge: Texas Instruments Device 8888 (rev 01)
0001:01:00.0 Non-VGA unclassified device: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)

And some further errors:

debian@arm:/boot$ sudo lspci -s 01:00.0 -v
0000:01:00.0 Non-VGA unclassified device: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
!!! Invalid class 0000 for header type 01
Flags: fast devsel
Bus: primary=00, secondary=00, subordinate=00, sec-latency=0
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable- Count=1/4 Maskable+ 64bit+
Capabilities: [68] Express Upstream Port, MSI 00
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch

  • 0
    Expert 1790 points
    Some of my message got deleted below the paste for the URL. The comment that is supposed to be prior to the dmesg dump is that it looks like the RC is only expecting to see end points connected to it and doesn't like seeing a switch.

    I'm looking for suggestions on how to allow the switch to be properly detected by the RC.

    A colleague using a PEX 8606 on an Intel solution has indicated that the switch ports will enumerate and can be seen with the lspci when the switch is properly detected.

    Thanks in advance for any suggestions.
  • 0 in reply to Chris Welch66
    Expert 1790 points

    It looks like a bug in the driver.  It is not reading the correct PCIe header type from the switch.  In dmesg it is reporting the header type for the switch as type 0 which is an end point:

    [    1.850353] PCI: bus0: Fast back to back transfers disabled

    [    1.850518] pci 0000:01:00.0: [10b5:8606] type 00 class 0x060400

    [    1.850547] pci 0000:01:00.0: ignoring class 0x060400 (doesn't match header type 00)

    [    1.850794] PCI: bus1: Fast back to back transfers disabled

    It looks like an index or alignment problem, because the switch does have a header type of 1 (class type in blue header type in yellow):

    debian@arm:~$ lspci -s 0000:01:00.0 -xx

    0000:01:00.0 Non-VGA unclassified device: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)

    00: b5 10 06 86 40 01 10 00 ba 00 04 06 10 00 01 00

    10: 00 00 00 00 00 00 00 00 00 00 00 00 f1 01 00 00

    20: f0 ff 00 00 f1 ff 01 00 00 00 00 00 00 00 00 00

    30: 00 00 00 00 40 00 00 00 00 00 00 00 00 01 00 00

    I'm suspecting this code has never been used with a subtended switch. Looking into the driver code.

  • 0 in reply to Chris Welch66
    Expert 1790 points
    Driver code looks good. It is a simple index into the PCI configuration header to pick up the header type.

    Switch configuration header does have correct driver type, so not sure why it doesn't get picked up correctly by the driver. Tried slowing the speed to GEN1, but exact same result. Been browsing through the switch configuration registers and everything looks good so far.

    Any suggestions would be appreciated.
  • 0 in reply to Chris Welch66
    Expert 1790 points
    Problem is that the PEX 8606 part does not allow access on non aligned 32 bit boundaries. A similar problem can be found in the discussion here:

    www.spinics.net/.../msg45095.html

    The PEX 8606 header type is read from PCI driver probe.c using byte access at 0xe and always gets back 0x00. Changing this to a 32 bit access at 0xc got the correct header type value of 0x01.

    I confirmed this on the command line using setpci to do byte and 32 bit aligned access and byte access only worked on 32 bit aligned addresses. Everything else returns 0.

    I've modified probe.c in the kernel (4.4.49) to handle this, but suspect there could be other problems due to non aligned access.
  • 0 in reply to Chris Welch66
    Expert 1790 points
    This looks like it might be related to errata i870, but our kernel has the patch for pci-dra7xx.c applied.
  • 0 in reply to Chris Welch66
    TI__Guru**** 170625 points
    Hi Chris,

    Thanks for sharing the solution.

    Regards,
    Pavel