SK-AM69: SK-AM69: PCIe Address Ranges on AM69

Part Number: SK-AM69
Other Parts Discussed in Thread: AM69

Following on from:

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1475274/sk-am69-pcie-address-ranges-on-am69

Appologies for the delayed response, back looking at this now.

I can now see the patches you refer to have been included in newer kernels and the AM69 can now support devices which require large 32bit non-prefetchable BARs (nearly upto 4GB):

ranges = <0x01000000 0x00 0x00001000 0x40 0x00001000 0x00 0x00100000>, /* IO (1 MB) */ 

         <0x02000000 0x00 0x00101000 0x40 0x00101000 0x00 0xffeff000>; /* 32-bit Non-Prefetchable MEM (4 GB - 1 MB - 4 KB) */

This is great, but what about devices with 64bit pre-fetchable BARs ?
If I was to split this address region up into 3 regions:

I/O  1MB

32bit Non-prefetch 2GB - 1MB - 4KB

64bit Pre-fetch 2GB

Would the hardware support this ?

I think the Cadence IP and Upstream linux driver should support this, and I can see a more standard setup in another SoC which uses the Cadence IP (but not a Ti Part) the "sophgo,sg2044" here is its 'ranges' entry from the PCIe node of its device tree:

 ranges = <0x01000000 0x0  0x00000000  0x48 0x10000000  0x0 0x00200000>, /* IO */
          <0x42000000 0x0  0x10000000  0x0  0x10000000  0x0 0x04000000>, /* 32bit Prefetchable */
          <0x02000000 0x0  0x14000000  0x0  0x14000000  0x0 0x04000000>, /* 32bit */
          <0x43000000 0x4a 0x00000000  0x4a 0x00000000  0x2 0x00000000>, /* 64bit */
          <0x03000000 0x49 0x00000000  0x49 0x00000000  0x1 0x00000000>; /* 64bit Prefetchable */

All bases are covered, so all devices should enumerate if they can fit in the sizes allocated.

 

 

 

  • Hi Neil,

    Should be possible to define different ranges.

    Regards,

    Takuma

  • Nice it works:

    Patch (just did pcie2 for now):

    diff --git a/arch/arm64/boot/dts/ti/k3-j784s4-j742s2-main-common.dtsi b/arch/arm64/boot/dts/ti/k3-j784s4-j742s2-main-common.dtsi
    index 9cc0901d58fb..c1df31d33f99 100644
    --- a/arch/arm64/boot/dts/ti/k3-j784s4-j742s2-main-common.dtsi
    +++ b/arch/arm64/boot/dts/ti/k3-j784s4-j742s2-main-common.dtsi
    @@ -1120,11 +1120,12 @@ pcie1_rc: pcie@2910000 {
             vendor-id = <0x104c>;
             device-id = <0xb012>;
             msi-map = <0x0 &gic_its 0x10000 0x10000>;
             dma-coherent;
             ranges = <0x01000000 0x00 0x00001000 0x41 0x00001000 0x00 0x00100000>, /* IO (1 MB) */
    -             <0x02000000 0x00 0x00101000 0x41 0x00101000 0x00 0xffeff000>; /* 32-bit Non-Prefetchable MEM (4 GB - 1 MB - 4 KB) */
    +             <0x02000000 0x00 0x00101000 0x41 0x00101000 0x00 0x7feff000>, /* 32-bit Non-Prefetchable MEM (2 GB - 1 MB - 4 KB) */
    +             <0x43000000 0x00 0x80000000 0x41 0x80000000 0x00 0x80000000>; /* 64-bit Prefetchable MEM (2 GB) */
             dma-ranges = <0x02000000 0x0 0x0 0x0 0x0 0x10000 0x0>;
             status = "disabled";
         };
     
         serdes_wiz0: wiz@5060000 {

    This gives:

    [    3.035198] j721e-pcie 2910000.pcie: host bridge /bus@100000/pcie@2910000 ranges:
    [    3.042689] j721e-pcie 2910000.pcie:       IO 0x4100001000..0x4100100fff -> 0x0000001000
    [    3.050780] j721e-pcie 2910000.pcie:      MEM 0x4100101000..0x417fffffff -> 0x0000101000
    [    3.058863] j721e-pcie 2910000.pcie:      MEM 0x4180000000..0x41ffffffff -> 0x0080000000
    [    3.066948] j721e-pcie 2910000.pcie:   IB MEM 0x0000000000..0xffffffffffff -> 0x0000000000

    And now my 64bit BAR enumerates:

    0001:01:00.0 Processing accelerators: Blaize, Inc Xplorer X1600 (rev 01)
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 255
        Region 0: Memory at 4180000000 (64-bit, prefetchable) [disabled] [size=512M]
        Region 2: Memory at 4101000000 (32-bit, non-prefetchable) [disabled] [size=16M]
        Region 3: Memory at 4100800000 (32-bit, non-prefetchable) [disabled] [size=4M]

    Is 4GB the limit for each PCIe controller or could we go bigger ? I have a device with an 8GB BAR0 size so would need >8GB to fit it in.

  • Hi Neil,

    4GB will be the limit.

    Regards,

    Takuma