• State Resolved
  • Locked Locked
  • Replies 4 replies
  • Subscribers 99 subscribers
  • Views 1605 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM5718: AM5718: PCIe not working on custom board

Robert Finger
Genius 17527 points
Part Number: AM5718


Tool/software: Linux

Team,

A customer of mine has issues to get PCIe RC correctly working on his own board. On the PCIe bus there is a FPGA which should get enumerated. But during Linux boot there are PCIe errors:

[    0.388807] libphy: mdio_driver_register: phy-bcm-ns2-pci
[    0.390024] pinctrl-single 4a003400.pinmux: 282 pins at pa fc003400 size 1128
[    0.398809] OF: PCI: host bridge /ocp/axi@0/pcie@51000000 ranges:
[    0.398819] OF: PCI:   No bus range found for /ocp/axi@0/pcie@51000000, using [bus 00-ff]
[    0.398845] OF: PCI:    IO 0x20003000..0x20012fff -> 0x00000000
[    0.398861] OF: PCI:   MEM 0x20013000..0x2fffffff -> 0x20013000
[    0.499853] dra7-pcie 51000000.pcie: link up
[    0.499992] dra7-pcie 51000000.pcie: PCI host bridge to bus 0000:00
[    0.500002] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.500011] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    0.500018] pci_bus 0000:00: root bus resource [mem 0x20013000-0x2fffffff]
[    0.500382] PCI: bus0: Fast back to back transfers disabled
[    0.500709] pci 0000:01:00.0: [Firmware Bug]: reg 0x1c: invalid BAR (can't size)
[    0.500742] pci 0000:01:00.0: [Firmware Bug]: reg 0x20: invalid BAR (can't size)
[    0.500774] pci 0000:01:00.0: [Firmware Bug]: reg 0x24: invalid BAR (can't size)
[    0.513841] PCI: bus1: Fast back to back transfers disabled
[    0.513898] irq: no irq domain found for /ocp/axi@0/pcie@51000000/interrupt-controller !
[    0.513942] irq: no irq domain found for /ocp/axi@0/pcie@51000000/interrupt-controller !
[    0.513981] pci 0000:00:00.0: BAR 8: assigned [mem 0x24000000-0x2fffffff]
[    0.513990] pci 0000:00:00.0: BAR 0: assigned [mem 0x20100000-0x201fffff]
[    0.514001] pci 0000:00:00.0: BAR 1: assigned [mem 0x20020000-0x2002ffff]
[    0.514014] pci 0000:01:00.0: BAR 0: assigned [mem 0x28000000-0x2fffffff]
[    0.514033] pci 0000:01:00.0: BAR 1: assigned [mem 0x24000000-0x24ffffff]
[    0.514051] pci 0000:01:00.0: BAR 2: assigned [mem 0x25000000-0x251fffff]
[    0.514070] pci 0000:00:00.0: PCI bridge to [bus 01]
[    0.514079] pci 0000:00:00.0:   bridge window [mem 0x24000000-0x2fffffff]
[    0.514276] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt
[    0.514285] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[    0.514732] OF: PCI: host bridge /ocp/axi@1/pcie@51800000 ranges:
[    0.514740] OF: PCI:   No bus range found for /ocp/axi@1/pcie@51800000, using [bus 00-ff]
[    0.514763] OF: PCI:    IO 0x30003000..0x30012fff -> 0x00000000
[    0.514779] OF: PCI:   MEM 0x30013000..0x3fffffff -> 0x30013000
[    1.515920] dra7-pcie 51800000.pcie: phy link never came up
[    1.516055] dra7-pcie 51800000.pcie: PCI host bridge to bus 0001:00

What might cause such a "invalid BAR" error message?

When doing a "lspci", the Mem bit is not set:

root@beaglebone:~# lspci -s 01:00.0 -vv
0000:01:00.0 Bridge: xxxxxxxxx
        Subsystem: Device 00d8:5a91
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 0
        Region 0: Memory at 28000000 (32-bit, non-prefetchable) [disabled] [size=128M]
        Region 1: Memory at 24000000 (32-bit, non-prefetchable) [disabled] [size=16M]
        Region 2: Memory at 25000000 (32-bit, non-prefetchable) [disabled] [size=2M]
        Region 3: I/O ports at <ignored> [disabled]
        Region 4: I/O ports at <ignored> [disabled]
        Region 5: I/O ports at <ignored> [disabled]
        Capabilities: [50] MSI: Enable- Count=1/4 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
...
lspci: Unable to load libkmod resources: error -12

But when manually setting the mem bit, the FPGA can be accessed via PCIe.Could this be a Linux Kernel config issue (mem bit not set)?

Thanks,
  Robert

  • 0
    TI__Guru**** 161600 points
    Hi Robert,

    I've notified the PCIe experts. Their feedback will be psoted here.

    Best Regards,
    Yordan
  • 0 in reply to Yordan Kovachev
    TI__Guru* 95265 points
    Hi,

    What version of the Linux kernel runs on it? Is this TI AM571x IDK EVM? And how the FPGA is connected to PCIE RC?

    It seems that the FPGA puts some wrong value at BAR locations 0x1c, 0x20 and 0x24. Are you able to provide a dump with "sudo lspci -xxx -s 01:00.0"?

    Regards, Eric
  • 0 in reply to lding
    Prodigy 10 points

    Hi,

    you are right, in the FPGA Registers where wrong values.

    I have corrected it and the "can't size" error message dissapears, but the Mem bit is still not set:

    root@beaglebone:~# lspci -xxx -s 01:00.0

    0000:01:00.0 Bridge: MEN Mikro Elektronik Multifunction IP core

    00: 88 1a 45 4d 40 01 10 00 00 00 80 06 10 00 00 00

    10: 00 00 00 28 00 00 00 24 00 00 00 22 00 00 00 00

    20: 00 00 00 00 00 00 00 00 00 00 00 00 d8 00 91 5a

    30: 00 00 00 00 50 00 00 00 00 00 00 00 00 01 00 00

    40: 00 00 00 00 60 61 00 02 00 00 00 00 00 00 00 00

    50: 05 78 84 00 00 00 00 00 00 00 00 00 00 00 00 00

    60: 00 00 00 00 00 00 00 00 11 78 00 00 00 00 00 00

    70: 00 00 00 00 00 00 00 00 01 80 03 00 08 00 00 00

    80: 10 00 12 00 01 80 00 00 10 28 00 00 11 f4 03 01

    90: 40 00 11 10 00 00 04 00 00 00 00 00 00 00 00 00

    a0: 00 00 00 00 10 00 00 00 00 00 00 00 00 00 00 00

    b0: 01 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00

    c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

    d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

    e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

    f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

    root@beaglebone:~# uname -r
    4.9.0-rc8-00010-g562b991

    It is not the  TI AM571x IDK EVM, we did our own board.

    FPGA is connected directly via PCIe x1 to the PCIe[0] port of the CPU.

    No switch is used.

    Regards

    Michael

  • 0 in reply to Michael Miehling
    TI__Guru* 95265 points

    Michael,

    Do you use the same FPGA for the test? Previously the device is Device 00d8:5a91, this time the device is 88 1a 45 4d .

    From 40 01 10 00, the memory access bit (bit 1) is still not set on FPGA side. You need to set this bit to enable the device to respond to memory accesses (40 01 10 00 ====>60 01 10 00). Then you should see: "Control: I/O- Mem+ BusMaster+" when enumerated.

    Regards, Eric