PROCESSOR-SDK-AM64X: PCIe Boot Mode and RootFS Loading on AM64x

Part Number: PROCESSOR-SDK-AM64X


Tool/software:

Hello TI Support Team,

I am currently working on booting the AM64x EVM as a PCIe endpoint and performing DFU via PCIe. I have been following the U-Boot PCIe DFU procedure described in this example, but I am encountering issues when trying to load subsequent images (e.g., the root filesystem).

Here is a summary of what I have done so far:

  • U-Boot SPL DFU over PCIe:
    • I am using AM64x EVM as the PCIe EP and J784s4 EVM as the PCIe RC.
    • I can successfully load tiboot3.bin, tispl.bin, and u-boot.img into the endpoint using the ./pcie_boot_copy tool from the root complex.
    • The SPL prints messages indicating that the images were received and authentication passed.
    • The PCIe link works during SPL, but as soon as U-Boot main (u-boot.img) starts, the PCIe link drops and does not come back.

  • U-Boot configuration:
CONFIG_PCI=y
CONFIG_PCI_ENDPOINT=y
CONFIG_SPL_PCI=y
CONFIG_SPL_PCI_ENDPOINT=y
CONFIG_CMD_PCI=y
# CONFIG_DM_PCI_COMPAT is not set
CONFIG_DM=y

CONFIG_PCI_DFU=y
CONFIG_SPL_PCI_DFU=y
CONFIG_PCI_DFU_SPL_LOAD_FIT_ADDRESS=0x81000000
CONFIG_PCI_DFU_BAR_SIZE=0x400000
CONFIG_PCI_DFU_VENDOR_ID=0x104c
CONFIG_PCI_DFU_DEVICE_ID=0xb010
CONFIG_PCI_DFU_BOOT_PHASE="tiboot3.bin"
CONFIG_PCI_DFU_MAGIC_WORD=0xdeadbeef
CONFIG_PCIE_CDNS_TI_EP=y
CONFIG_SPL_RAM_SUPPORT=y
CONFIG_SPL_RAM_DEVICE=y
CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR=0x800
CONFIG_ENV_SIZE=0x40000
CONFIG_ENV_OFFSET=0x3400
  • Device Tree:
    • I have attempted to configure k3-am642-evm.dts with a PCIe endpoint node:

&pcie0_rc {
status = "disabled";
reset-gpios = <&exp1 5 GPIO_ACTIVE_HIGH>;
phys = <&serdes0_pcie_link>;
phy-names = "pcie-phy";
num-lanes = <1>;
};

&cbass_main {
#address-cells = <2>;
#size-cells = <2>;
interrupt-parent = <&gic500>;

pcie0_ep: pcie-ep@f102000 {
compatible = "ti,am64-pcie-ep", "ti,j721e-pcie-ep";
reg = <0x00 0x0f102000 0x00 0x1000>,
<0x00 0x0f100000 0x00 0x400>,
<0x00 0x0d000000 0x00 0x00800000>,
<0x00 0x68000000 0x00 0x08000000>;
reg-names = "intd_cfg", "user_cfg", "reg", "mem";
interrupt-names = "link_state";
interrupts = <GIC_SPI 203 IRQ_TYPE_EDGE_RISING>;
max-link-speed = <2>;
num-lanes = <1>;
power-domains = <&k3_pds 114 TI_SCI_PD_EXCLUSIVE>;
clocks = <&k3_clks 114 0>;
clock-names = "fck";
max-functions = /bits/ 8 <1>;
phys = <&serdes0_pcie_link>;
phy-names = "pcie-phy";
bootph-all;
ti,syscon-pcie-ctrl = <&main_conf 0x4070>;
status = "okay";
};
};

Despite this configuration, once U-Boot main starts, the PCIe link drops and the endpoint disappears from the root complex.

  • Current issue:
    • I can load the initial bootloaders via PCIe DFU (SPL), but once U-Boot main runs, the PCIe link disappears.
    • I need to send a larger image (rootfs.tar.gz) to the endpoint memory over PCIe, but cannot maintain the link long enough.

  • Request:

Could you please advise:

The correct way to keep the PCIe link active in U-Boot main when running as an endpoint?

How to safely load the root filesystem (rootfs.tar.gz) into the endpoint memory over PCIe, considering PCIe Boot mechanism?

Any guidance, example commands, or patches would be greatly appreciated.

Thank you in advance.

André

  • Hi Andre,

    Interestingly I was not aware that the sdk supports the PCIe DFU boot mode. Please allow me some time to understand the feature and look into your issue. I am currently working on a customer issue so it might take some time to respond here. 

  • Hi Andre,

    The log below shows the PCIe boot procedure on the RC side. Please ensure you have done the PCI rescan process.

    # ./a.out 0x10200000 tiboot3.bin 
    bootfilename: tiboot3.bin
    bar1_address: 0x10200000
    load_addr: 0x70000000
    load_addr_offset: 0x1000
    start_addr_offset: 0x1bcfe0
    try to open /dev/mem.
    /dev/mem opened.
    tiboot3.bin opened.
    image_size: 527542
    main: image_size: 527542
    main: map_size: 4194304
    map_base: 0xffff80200000
    Read image of 527542 bytes
    Writing image to memory
    done.

    # echo 1 > /sys/bus/pci/devices/0000\:01\:00.0/remove
    # echo 1 > /sys/bus/pci/devices/0000\:00\:00.0/rescan
    [   65.908998] pci 0000:01:00.0: [104c:b010] type 00 class 0x050000 PCIe Endpoint
    [   65.916290] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x000fffff]
    [   65.922234] pci 0000:01:00.0: BAR 1 [mem 0x02000000-0x021fffff pref]
    [   65.928630] pci 0000:01:00.0: BAR 2 [mem 0x04000000-0x05ffffff 64bit pref]
    [   65.935542] pci 0000:01:00.0: BAR 4 [mem 0x06000000-0x07ffffff 64bit pref]
    [   65.942921] pci 0000:01:00.0: supports D1
    [   65.946927] pci 0000:01:00.0: PME# supported from D0 D1 D3hot
    [   65.959048] pci 0000:01:00.0: BAR 2 [mem 0x12000000-0x13ffffff 64bit pref]: assigned
    [   65.966815] pci 0000:01:00.0: BAR 4 [mem 0x14000000-0x15ffffff 64bit pref]: assigned
    [   65.974581] pci 0000:01:00.0: BAR 1 [mem 0x10200000-0x103fffff pref]: assigned
    [   65.981810] pci 0000:01:00.0: BAR 0 [mem 0x10100000-0x101fffff]: assigned
    [   65.988771] pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22

    # ./a.out 0x10200000 tiboot3.bin 
    bootfilename: tiboot3.bin
    bar1_address: 0x10200000
    load_addr: 0x70000000
    load_addr_offset: 0x1000
    start_addr_offset: 0x1bcfe0
    try to open /dev/mem.
    /dev/mem opened.
    tiboot3.bin opened.
    image_size: 527542
    main: image_size: 527542
    main: map_size: 4194304
    map_base: 0xffffa1000000
    Read image of 527542 bytes
    Writing image to memory
    done.

    # echo 1 > /sys/bus/pci/devices/0000\:01\:00.0/remove
    # echo 1 > /sys/bus/pci/devices/0000\:00\:00.0/rescan
    [   78.152936] pci 0000:01:00.0: [104c:b010] type 00 class 0x050000 PCIe Endpoint
    [   78.160208] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x003fffff pref]
    [   78.167139] pci 0000:01:00.0: supports D1
    [   78.171145] pci 0000:01:00.0: PME# supported from D0 D1 D3hot
    [   78.183047] pci 0000:01:00.0: BAR 0 [mem 0x12000000-0x123fffff pref]: assigned
    [   78.190409] pci-endpoint-test 0000:01:00.0: enabling device (0000 -> 0002)

    # ./a.out 0x12000000 tispl.bin   
    bootfilename: tispl.bin
    bar1_address: 0x12000000
    load_addr: 0xdeadbeef
    load_addr_offset: 0x0
    start_addr_offset: 0x3ffffc
    try to open /dev/mem.
    /dev/mem opened.
    tispl.bin opened.
    image_size: 938375
    main: image_size: 938375
    main: map_size: 4194304
    map_base: 0xffffa4200000
    Read image of 938375 bytes
    Writing image to memory
    done.

    # echo 1 > /sys/bus/pci/devices/0000\:01\:00.0/remove
    # echo 1 > /sys/bus/pci/devices/0000\:00\:00.0/rescan
    [  108.360957] pci 0000:01:00.0: [104c:b010] type 00 class 0x050000 PCIe Endpoint
    [  108.368233] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x003fffff pref]
    [  108.375208] pci 0000:01:00.0: supports D1
    [  108.379213] pci 0000:01:00.0: PME# supported from D0 D1 D3hot
    [  108.391052] pci 0000:01:00.0: BAR 0 [mem 0x12000000-0x123fffff pref]: assigned
    [  108.398413] pci-endpoint-test 0000:01:00.0: enabling device (0000 -> 0002)

    # ./a.out 0x12000000 u-boot.img 
    bootfilename: u-boot.img
    bar1_address: 0x12000000
    load_addr: 0xdeadbeef
    load_addr_offset: 0x0
    start_addr_offset: 0x3ffffc
    try to open /dev/mem.
    /dev/mem opened.
    u-boot.img opened.
    image_size: 1655147
    main: image_size: 1655147
    main: map_size: 4194304
    map_base: 0xffff80c00000
    Read image of 1655147 bytes
    Writing image to memory
    done.

  • Hi Bin Lui,


    Thank you for your detailed response and confirmation regarding the PCIe boot procedure.
    The rescan process and image loading steps for tiboot3.bin, tispl.bin, and u-boot.img are now clear and working as expected on my setup.

    However, I still need clarification on the next step — specifically, how to load a root filesystem image (rootfs.tar.gz) into the AM64x endpoint memory over PCIe once U-Boot is running.

    Could you please advise:

    1. Whether there is a supported or recommended mechanism to transfer a large image such as the root filesystem (via the same PCIe mechanism)?

    2. If PCIe DFU can be reused for this purpose, what address or BAR region should be targeted?

    3. If not, what is the correct sequence or configuration required to maintain the PCIe link during or after U-Boot, so that I can access the endpoint memory from the root complex?

    Any additional guidance, documentation, or example scripts for transferring the root filesystem over PCIe would be very helpful.

    Thanks for your support.

    Regards,
    André

  • Hi Andre,

    PCI DFU is a very new feature added in U-Boot, I have not found any instruction related to loading images from U-Boot command line usinf PCI DFU. But I will continue working on it and let you know.

  • Hi Andre,

    Currently the U-Boot DFU command doesn't support download from PCIe interface yet. So other than boot into U-Boot prompt, you currently won't be able to download kernel or rootfs images through the PCIe interface. You would have to use any other supported interface to do so.

  • Hello Bin Liu,

    Thank you very much for the clarification and support regarding PCIe DFU.
    If there are any updates in the future regarding PCIe DFU support for rootfs transfer, please let me know in the meanwhile.

    Thanks!

    Regards,
    André

  • Hi Andre,

    Currently the sw dev team doesn't have a plan to enable this feature in the near future, for example, for the next 6 months. But I will take a note to unlock and update this thread if there is any planning change down the road.