This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VEN-Q1: Pcitest fail issue between two J722s soc connection

Part Number: TDA4VEN-Q1

Hi TI team,

 

We connect two TDA4VEN soc with 1-lane PCIe3.0 on our customized board. And the pcitest failed after the PCIe connection between two soc link up. Here are logs of RC and EP: 

RC: 

image.png

 

EP:

image.png

There are many fails when executing interrupt tests. And the pcitest process is stuck when executing read test. Pcitest binary and pcitest.sh shell script we used are located at <linux-kernel-source>/tools/pci in SDK1101. 

Please help to validate the PCIe link between J722s EVM boards, thank you.

Regards,

Mingjian

  • Hi Mingjian,

    The failing interrupts are expected.

    • Legacy interrupt - We removed due to a silicon errata i2094 which was first seen on J721E, but seen on other J7x devices as well: https://www.ti.com/lit/er/sprz455f/sprz455f.pdf?ts=1764885576311&ref_url=https%253A%252F%252Fwww.google.com%252F 
    • MSI interrupt - If the SDK documentation is followed, then the default example only enables the first two MSI interrupts. Specifically, below two lines:
      root@evm:/sys/kernel/config/pci_ep# echo 2 > functions/pci_epf_test/func1/msi_interrupts
      root@evm:/sys/kernel/config/pci_ep# echo 2 > functions/pci_epf_test/func1/msix_interrupts
      Increasing the number of MSI/MSI-X interrupts on EP-side should increase the number of passing interrupts.

    As for the read test, it is unexpected that the test is stuck with read test. Could you try directly interacting with the pcitest binary instead of the bash script for troubleshooting? The read test can be initiated with a command like so: pcitest -d -r -s <size-in-bytes>

    Regards,

    Takuma

  • HI  Takuma:
    As shown in the attachment, we first configure on the EP side using setup-epf-func-VEN.sh, and then test on the RC side using pcitest.sh.

    mount -t configfs none /sys/kernel/config
    
    cd /sys/kernel/config/pci_ep/
    mkdir functions/pci_epf_test/func1
    echo 0x104c > functions/pci_epf_test/func1/vendorid
    echo 0xb010 > functions/pci_epf_test/func1/deviceid
    echo 2 > functions/pci_epf_test/func1/msi_interrupts
    echo 2 > functions/pci_epf_test/func1/msix_interrupts
    ln -s functions/pci_epf_test/func1 controllers/f102000.pcie-ep/
    
    
    echo 1 > controllers/f102000.pcie-ep/start
    
    
    #cd /sys/kernel/config/pci_ep/
    #mkdir functions/pci_epf_test/func2
    #echo 0x104c > functions/pci_epf_test/func2/vendorid
    #echo 0xb010 > functions/pci_epf_test/func2/deviceid
    #echo 2 > functions/pci_epf_test/func2/msi_interrupts
    #echo 2 > functions/pci_epf_test/func2/msix_interrupts
    #ln -s functions/pci_epf_test/func2 controllers/f102000.pcie-ep/
    #
    #cd /sys/kernel/config/pci_ep/
    #mkdir functions/pci_epf_test/func3
    #echo 0x104c > functions/pci_epf_test/func3/vendorid
    #echo 0xb010 > functions/pci_epf_test/func3/deviceid
    #echo 2 > functions/pci_epf_test/func3/msi_interrupts
    #echo 2 > functions/pci_epf_test/func3/msix_interrupts
    #ln -s functions/pci_epf_test/func3 controllers/f102000.pcie-ep/
    #
    #cd /sys/kernel/config/pci_ep/
    #mkdir functions/pci_epf_test/func4
    #echo 0x104c > functions/pci_epf_test/func4/vendorid
    #echo 0xb010 > functions/pci_epf_test/func4/deviceid
    #echo 2 > functions/pci_epf_test/func4/msi_interrupts
    #echo 2 > functions/pci_epf_test/func4/msix_interrupts
    #ln -s functions/pci_epf_test/func4 controllers/f102000.pcie-ep/
    #
    #cd /sys/kernel/config/pci_ep/
    #mkdir functions/pci_epf_test/func5
    #echo 0x104c > functions/pci_epf_test/func5/vendorid
    #echo 0xb010 > functions/pci_epf_test/func5/deviceid
    #echo 2 > functions/pci_epf_test/func5/msi_interrupts
    #echo 2 > functions/pci_epf_test/func5/msix_interrupts
    #ln -s functions/pci_epf_test/func5 controllers/f102000.pcie-ep/
    #
    #cd /sys/kernel/config/pci_ep/
    #mkdir functions/pci_epf_test/func6
    #echo 0x104c > functions/pci_epf_test/func6/vendorid
    #echo 0xb010 > functions/pci_epf_test/func6/deviceid
    #echo 2 > functions/pci_epf_test/func6/msi_interrupts
    #echo 2 > functions/pci_epf_test/func6/msix_interrupts
    #ln -s functions/pci_epf_test/func6 controllers/f102000.pcie-ep/
    #!/bin/sh
    # SPDX-License-Identifier: GPL-2.0
    
    echo 1 > /sys/bus/pci/rescan
    echo "BAR tests"
    echo
    
    bar=0
    
    while [ $bar -lt 6 ]
    do
    	pcitest -b $bar
    	bar=`expr $bar + 1`
    done
    echo
    
    echo "Interrupt tests"
    echo
    
    #SET IRQ TYPE TO LEGACY:         NOT OKAY
    #pcitest -i 0
    ##LEGACY IRQ:     NOT OKAY   EP:pci_epf_test pci_epf_test.0: PCIE:pci_epf_test_raise_irq ===== 0 / 0
    #pcitest -l
    #
    #SET IRQ TYPE TO MSI:            OKAY
    pcitest -i 1 
    msi=1
    # pci_epf_test pci_epf_test.0: PCIE:pci_epf_test_raise_irq ===== 1 / 0    pcitest -m 1
    # pci_epf_test pci_epf_test.0: Invalid MSI IRQ number 2 / 0    pcitest -m 2
    while [ $msi -lt 3 ]
    do
            pcitest -m $msi
            msi=`expr $msi + 1`
    done
    echo
    
    #SET IRQ TYPE TO MSI-X:          OKAY
    
    pcitest -i 2
    msix=1
    
    while [ $msix -lt 3 ]
    do
            pcitest -x $msix
            msix=`expr $msix + 1`
    done
    echo
    
    echo "Read Tests"
    echo
    
    pcitest -i 1
    
    pcitest -r -s 1
    pcitest -r -s 1024
    pcitest -r -s 1025
    pcitest -r -s 1024000
    pcitest -r -s 1024001
    
    pcitest -d -r -s 1
    pcitest -d -r -s 1024
    pcitest -d -r -s 1025
    pcitest -d -r -s 1024000
    pcitest -d -r -s 1024001
    echo
    
    echo "Write Tests"
    echo
    
    pcitest -w -s 1
    pcitest -w -s 1024
    pcitest -w -s 1025
    pcitest -w -s 1024000
    pcitest -w -s 1024001
    
    pcitest -d -w -s 1
    pcitest -d -w -s 1024
    pcitest -d -w -s 1025
    pcitest -d -w -s 1024000
    pcitest -d -w -s 1024001
    echo
    
    echo "Copy Tests"
    echo
    
    pcitest -c -s 1
    pcitest -c -s 1024
    pcitest -c -s 1025
    pcitest -c -s 1024000
    pcitest -c -s 1024001
    
    pcitest -d -c -s 1
    pcitest -d -c -s 1024
    pcitest -d -c -s 1025
    pcitest -d -c -s 1024000
    pcitest -d -c -s 1024001
    echo
    

    During debugging of data read and write, it was found that in pci_endpoint_test.c, in the function static irqreturn_t pci_endpoint_test_irqhandler(int irq, void *dev_id), the register value reg read from PCI_ENDPOINT_TEST_STATUS was 0, which caused the code to not reach complete(&test->irq_raised); Furthermore, during testing, the read and write successes happen randomly, but most of the time failures occur.    

    Please help explain what causes the PCI_ENDPOINT_TEST_STATUS register to read a value of 0 during the data read/write process, and who sets it? RC or EP?

  •  Change all instances of wait_for_compilete in the code to wait_for_compilete_timeout, then perform PCIe read and write tests. The results are as follows, with OKAY and NOT OKAY appearing randomly each time.

    ./pcitest.sh
    BAR tests
    BAR0:           OKAY
    BAR1:           OKAY
    BAR2:           OKAY
    BAR3:           OKAY
    BAR4:           OKAY
    BAR5:           OKAY
    Interrupt tests
    SET IRQ TYPE TO MSI:            OKAY
    MSI1:           OKAY
    MSI2:           OKAY
    SET IRQ TYPE TO MSI-X:          OKAY
    MSI-X1:         OKAY
    MSI-X2:         OKAY
    Read Tests
    SET IRQ TYPE TO MSI:            OKAY
    READ (      1 bytes):           NOT OKAY
    READ (   1024 bytes):           NOT OKAY
    READ (   1025 bytes):           NOT OKAY
    READ (1024000 bytes):           NOT OKAY
    READ (1024001 bytes):           OKAY
    READ (      1 bytes):           OKAY
    READ (   1024 bytes):           OKAY
    READ (   1025 bytes):           NOT OKAY
    READ (1024000 bytes):           OKAY
    READ (1024001 bytes):           NOT OKAY
    Write Tests
    WRITE (      1 bytes):          OKAY
    WRITE (   1024 bytes):          OKAY
    WRITE (   1025 bytes):          NOT OKAY
    WRITE (1024000 bytes):          NOT OKAY
    WRITE (1024001 bytes):          NOT OKAY
    WRITE (      1 bytes):          NOT OKAY
    WRITE (   1024 bytes):          NOT OKAY
    WRITE (   1025 bytes):          NOT OKAY
    WRITE (1024000 bytes):          NOT OKAY
    WRITE (1024001 bytes):          NOT OKAY
    Copy Tests
    COPY (      1 bytes):           NOT OKAY
    COPY (   1024 bytes):           NOT OKAY
    COPY (   1025 bytes):           OKAY
    COPY (1024000 bytes):           NOT OKAY
    COPY (1024001 bytes):           NOT OKAY
    COPY (      1 bytes):           NOT OKAY
    COPY (   1024 bytes):           OKAY
    COPY (   1025 bytes):           NOT OKAY
    COPY (1024000 bytes):           OKAY
    COPY (1024001 bytes):           OKAY
  • Hi Quansheng,

    Please help explain what causes the PCI_ENDPOINT_TEST_STATUS register to read a value of 0 during the data read/write process, and who sets it? RC or EP?

    Below is a description of PCI_ENDPOINT_TEST_STATUS. 

    * PCI_ENDPOINT_TEST_STATUS
    
    This register reflects the status of the PCI endpoint device.
    
    ========	==============================
    Bitfield	Description
    ========	==============================
    Bit 0		read success
    Bit 1		read fail
    Bit 2		write success
    Bit 3		write fail
    Bit 4		copy success
    Bit 5		copy fail
    Bit 6		IRQ raised
    Bit 7		source address is invalid
    Bit 8		destination address is invalid
    ========	==============================

    The PCI_ENDPOINT_TEST_STATUS bits are set within the pci-epf-test.c driver that will run on the EP-side. The pcitest binary that is used by the bash script should send out a command to pci-epf-test by writing to the PCI_ENDPOINT_TEST_COMMAND register. On EP-side, they are then handled by the cmd_handler to call various functions depending on the command.

    For troubleshooting, you could put some print statements below to see if the cmd_handler is getting called properly:

    https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/pci/endpoint/functions/pci-epf-test.c?h=ti-linux-6.1.y#n643 

    Regards,

    Takuma

  • Hi Takuma, 

    We added the print in pci-epf-test.c and pci-endpoint-test.c. Seems like EP did not got the correct command when RC write it corrrectly. 

    RC log:

    root@spartan_a:/data# ./pcitest.sh
    BAR tests
    
    BAR0:           OKAY
    BAR1:           OKAY
    BAR2:           OKAY
    BAR3:           OKAY
    BAR4:           OKAY
    BAR5:           OKAY
    
    Interrupt tests
    
    SET IRQ TYPE TO LEGACY:         NOT OKAY
    LEGACY IRQ:     NOT OKAY
    SET IRQ TYPE TO MSI:            OKAY
    MSI1:           OKAY
    MSI2:           NOT OKAY
    
    SET IRQ TYPE TO MSI-X:          OKAY
    MSI-X1:         NOT OKAY
    MSI-X2:         OKAY
    
    Read Tests
    
    SET IRQ TYPE TO MSI-X:          OKAY
    READ (      1 bytes):           OKAY
    READ (   1024 bytes):           OKAY
    READ (   1025 bytes):           OKAY
    READ (1024000 bytes):           OKAY
    READ (1024001 bytes):           OKAY
    
    Write Tests
    
    WRITE (      1 bytes):          OKAY
    WRITE (   1024 bytes):          OKAY
    WRITE (   1025 bytes):          OKAY
    WRITE (1024000 bytes):          NOT OKAY
    WRITE (1024001 bytes):          OKAY
    
    Copy Tests
    
    COPY (      1 bytes):           OKAY
    COPY (   1024 bytes):           OKAY
    COPY (   1025 bytes):           OKAY
    COPY (1024000 bytes):           OKAY
    COPY (1024001 bytes):           OKAY
    
    root@spartan_a:/data# dmesg | grep pci
    [    1.526503] j721e-pcie f102000.pcie: host bridge /pcie@f102000 ranges:
    [    1.536376] j721e-pcie f102000.pcie:       IO 0x0600001000..0x0600100fff -> 0x0000001000
    [    1.548232] j721e-pcie f102000.pcie:      MEM 0x0600101000..0x06ffffffff -> 0x0000101000
    [    1.564031] j721e-pcie f102000.pcie:   IB MEM 0x0000000000..0xffffffffffff -> 0x0000000000
    [    2.581081] j721e-pcie f102000.pcie: PCI host bridge to bus 0000:00
    [    2.587372] pci_bus 0000:00: root bus resource [bus 00-ff]
    [    2.592858] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0x1000-0x100fff])
    [    2.601895] pci_bus 0000:00: root bus resource [mem 0x600101000-0x6ffffffff] (bus address [0x00101000-0xffffffff])
    [    2.612258] pci 0000:00:00.0: [104c:b010] type 01 class 0x060400 PCIe Root Port
    [    2.619574] pci 0000:00:00.0: PCI bridge to [bus 00]
    [    2.624539] pci 0000:00:00.0:   bridge window [io  0x0000-0x0fff]
    [    2.630627] pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
    [    2.637414] pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
    [    2.645205] pci 0000:00:00.0: supports D1
    [    2.649213] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
    [    2.657980] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
    [    2.666180] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
    [    2.672816] pci 0000:00:00.0: PCI bridge to [bus 01]
    [    2.677783] pci_bus 0000:00: resource 4 [io  0x0000-0xfffff]
    [    2.683438] pci_bus 0000:00: resource 5 [mem 0x600101000-0x6ffffffff]
    [    2.689984] pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
    [    2.696938] pcieport 0000:00:00.0: PME: Signaling with IRQ 513
    [    2.703064] pcieport 0000:00:00.0: AER: enabled with IRQ 513
    [   69.733641] pci 0000:01:00.0: [104c:b010] type 00 class 0xff0000 PCIe Endpoint
    [   69.740926] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x000000ff]
    [   69.746881] pci 0000:01:00.0: BAR 1 [mem 0x00000000-0x000001ff]
    [   69.752833] pci 0000:01:00.0: BAR 2 [mem 0x00000000-0x000003ff]
    [   69.758773] pci 0000:01:00.0: BAR 3 [mem 0x00000000-0x00003fff]
    [   69.764724] pci 0000:01:00.0: BAR 4 [mem 0x00000000-0x0001ffff]
    [   69.770662] pci 0000:01:00.0: BAR 5 [mem 0x00000000-0x000fffff]
    [   69.776874] pci 0000:01:00.0: supports D1
    [   69.780893] pci 0000:01:00.0: PME# supported from D0 D1 D3hot
    [   69.792910] pcieport 0000:00:00.0: bridge window [mem 0x600200000-0x6003fffff]: assigned
    [   69.801061] pci 0000:01:00.0: BAR 5 [mem 0x600200000-0x6002fffff]: assigned
    [   69.808061] pci 0000:01:00.0: BAR 4 [mem 0x600300000-0x60031ffff]: assigned
    [   69.815061] pci 0000:01:00.0: BAR 3 [mem 0x600320000-0x600323fff]: assigned
    [   69.822038] pci 0000:01:00.0: BAR 2 [mem 0x600324000-0x6003243ff]: assigned
    [   69.829025] pci 0000:01:00.0: BAR 1 [mem 0x600324400-0x6003245ff]: assigned
    [   69.835999] pci 0000:01:00.0: BAR 0 [mem 0x600324600-0x6003246ff]: assigned
    [   69.843200] pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
    [   69.849850] pci-endpoint-test 0000:01:00.0: enabling device (0000 -> 0002)
    [   86.880954] pci-endpoint-test 0000:01:00.0: Failed to get Legacy interrupt
    [   87.920157] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_msi_irq, write command 2
    [   88.692639] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_msi_irq, write command 2
    [   89.717115] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_msi_irq, write command 4
    [   90.736461] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_msi_irq, write command 4
    [   90.782716] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read start
    [   90.789479] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read, write command 16
    [   91.044624] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read start
    [   91.051383] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read, write command 16
    [   92.076298] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read start
    [   92.083080] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read, write command 16
    [   93.100352] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read start
    [   93.107526] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read, write command 16
    [   94.124990] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read start
    [   94.131958] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_read, write command 16
    [   95.149366] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_write start
    [   95.156210] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_write, write command 8
    [   95.292205] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_write start
    [   95.299076] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_write, write command 8
    [   96.332272] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_write start
    [   96.339133] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_write, write command 8
    [   97.356281] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_write start
    [   97.373150] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_write, write command 8
    [   98.412336] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_write start
    [   98.429151] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_write, write command 8
    [   99.468610] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_copy start
    [   99.475382] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_copy, write command 32
    [  100.492278] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_copy start
    [  100.499097] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_copy, write command 32
    [  100.716687] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_copy start
    [  100.723464] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_copy, write command 32
    [  101.740221] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_copy start
    [  101.757169] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_copy, write command 32
    [  102.797068] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_copy start
    [  102.814387] pci-endpoint-test 0000:01:00.0: pci_endpoint_test_copy, write command 32

    EP log: 

    ▒[   87.636515] pci_epf_test pci_epf_test.0: Got command from rc, 1
    [   87.642474] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_INTX,reg->irq_number=0
    [   87.652190] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_INTX,reg->irq_number=0
    [   88.632535] pci_epf_test pci_epf_test.0: Got command from rc, 2
    [   88.638489] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSI,reg->irq_number=1
    [   88.647104] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSI,reg->irq_number=1
    [   90.712558] pci_epf_test pci_epf_test.0: Got command from rc, 4
    [   90.724540] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=2
    [   90.733362] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=2
    [   90.964564] pci_epf_test pci_epf_test.0: Got command from rc, 16
    [   90.976536] pci_epf_test pci_epf_test.0: pci_epf_test_write start
    [   90.982727] pci_epf_test pci_epf_test.0: WRITE => Size: 1 B, DMA: NO, Time: 0.000000660 s, Rate: 1515 KB/s
    [   90.994523] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [   91.003303] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [   91.016581] pci_epf_test pci_epf_test.0: Got command from rc, 16
    [   91.022671] pci_epf_test pci_epf_test.0: pci_epf_test_write start
    [   91.028848] pci_epf_test pci_epf_test.0: WRITE => Size: 1024 B, DMA: NO, Time: 0.000004090 s, Rate: 250366 KB/s
    [   91.040515] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [   91.049285] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [   92.200514] pci_epf_test pci_epf_test.0: Got command from rc, 16
    [   92.206552] pci_epf_test pci_epf_test.0: pci_epf_test_write start
    [   92.212707] pci_epf_test pci_epf_test.0: WRITE => Size: 1025 B, DMA: NO, Time: 0.000004630 s, Rate: 221382 KB/s
    [   92.224519] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [   92.233229] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [   93.196553] pci_epf_test pci_epf_test.0: Got command from rc, 16
    [   93.202610] pci_epf_test pci_epf_test.0: pci_epf_test_write start
    [   93.222555] pci_epf_test pci_epf_test.0: WRITE => Size: 1024000 B, DMA: NO, Time: 0.003841720 s, Rate: 266547 KB/s
    [   93.235008] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [   93.243725] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [   94.204606] pci_epf_test pci_epf_test.0: Got command from rc, 16
    [   94.211792] pci_epf_test pci_epf_test.0: pci_epf_test_write start
    [   94.231867] pci_epf_test pci_epf_test.0: WRITE => Size: 1024001 B, DMA: NO, Time: 0.003861930 s, Rate: 265152 KB/s
    [   94.244342] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [   94.253058] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [   95.220538] pci_epf_test pci_epf_test.0: Got command from rc, 8
    [   95.226485] pci_epf_test pci_epf_test.0: pci_epf_test_read start
    [   95.232618] pci_epf_test pci_epf_test.0: READ => Size: 1 B, DMA: NO, Time: 0.000001905 s, Rate: 524 KB/s
    [   95.242139] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [   95.250858] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [   95.264521] pci_epf_test pci_epf_test.0: Got command from rc, 8
    [   95.270456] pci_epf_test pci_epf_test.0: pci_epf_test_read start
    [   95.276645] pci_epf_test pci_epf_test.0: READ => Size: 1024 B, DMA: NO, Time: 0.000145010 s, Rate: 7061 KB/s
    [   95.286525] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [   95.295238] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [   96.540514] pci_epf_test pci_epf_test.0: Got command from rc, 8
    [   96.546469] pci_epf_test pci_epf_test.0: pci_epf_test_read start
    [   96.552674] pci_epf_test pci_epf_test.0: READ => Size: 1025 B, DMA: NO, Time: 0.000144630 s, Rate: 7087 KB/s
    [   96.562547] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [   96.571265] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [   97.644547] pci_epf_test pci_epf_test.0: Got command from rc, 8
    [   97.650505] pci_epf_test pci_epf_test.0: pci_epf_test_read start
    [   97.801418] pci_epf_test pci_epf_test.0: READ => Size: 1024000 B, DMA: NO, Time: 0.144655190 s, Rate: 7078 KB/s
    [   97.812080] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [   97.820841] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [   99.640543] pci_epf_test pci_epf_test.0: Got command from rc, 32
    [   99.646597] pci_epf_test pci_epf_test.0: pci_epf_test_copy start
    [   99.652677] pci_epf_test pci_epf_test.0: COPY => Size: 1 B, DMA: NO, Time: 0.000004005 s, Rate: 249 KB/s
    [   99.662186] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [   99.670898] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [  100.644563] pci_epf_test pci_epf_test.0: Got command from rc, 32
    [  100.650602] pci_epf_test pci_epf_test.0: pci_epf_test_copy start
    [  100.656812] pci_epf_test pci_epf_test.0: COPY => Size: 1024 B, DMA: NO, Time: 0.000150290 s, Rate: 6813 KB/s
    [  100.666684] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [  100.675393] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [  100.688517] pci_epf_test pci_epf_test.0: Got command from rc, 32
    [  100.694553] pci_epf_test pci_epf_test.0: pci_epf_test_copy start
    [  100.700755] pci_epf_test pci_epf_test.0: COPY => Size: 1025 B, DMA: NO, Time: 0.000152195 s, Rate: 6734 KB/s
    [  100.710634] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [  100.719350] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [  101.792555] pci_epf_test pci_epf_test.0: Got command from rc, 32
    [  101.798638] pci_epf_test pci_epf_test.0: pci_epf_test_copy start
    [  101.953629] pci_epf_test pci_epf_test.0: COPY => Size: 1024000 B, DMA: NO, Time: 0.148787975 s, Rate: 6882 KB/s
    [  101.963795] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [  101.972519] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1
    [  103.264516] pci_epf_test pci_epf_test.0: Got command from rc, 32
    [  103.270552] pci_epf_test pci_epf_test.0: pci_epf_test_copy start
    [  103.425337] pci_epf_test pci_epf_test.0: COPY => Size: 1024001 B, DMA: NO, Time: 0.148673655 s, Rate: 6887 KB/s
    [  103.435495] pci_epf_test pci_epf_test.0: pci_epc_raise_irq start,PCI_IRQ_MSIX,reg->irq_number=1
    [  103.444210] pci_epf_test pci_epf_test.0: pci_epc_raise_irq finish,PCI_IRQ_MSIX,reg->irq_number=1

    The command printed from RC is using pci_endpoint_test_readl() after pci_endpoint_test_writel(). The read results are all correct. But sometimes EP could not get command from RC and did not enter the pci_endpoint_test_read() function. It caused the fail of RC's write test. The same issue happend in irq tests. RC should send  irq commands for 4 times but EP only received twice. Please help analyse the issue.

    Regards, 

    Mingjian 

  • Hi Mingjian,

    I noticed previous logs when read/writes were done, MSI interrupts were set. Newer logs seem to be using MSI-X for read/writes. Could you set MSI before read/write, collect logs from read/write, and then test setting up MSI-X then collecting same logs for comparison? The debug prints that you have put in will be useful for the comparison.

    I have recently heard of a customer issue that may be related to MSI vs MSI-X interrupt, so would be good to see if that is affecting the PCIe EP/RC example as well.

    Regards,

    Takuma 

  • Hi Takuma , 

    I tried using MSI-X instead of MSI, but seems that there is no difference. 

    On the other side , I think I found the root cause of read, write and copy test fail. When RC pci_endpoint_test driver wrote pci bar space , such as COMMAND, it sometimes took 0.5 second or longer then EP can read the COMMAND which RC wrote and go to deal with the COMMAND. Such delay also exists when EP wrote, RC read. So when EP finish the COMMAND function(read,write,copy), EP wrote SUCCESS to the status bit and raise irq to inform RC. But when RC receive irq, RC  could not  read the status bit as SUCCESS, so it return fail.

    To validate the issue above, I disabled the irq callback  and add  3s delay after RC send COMMAND to EP.  After this modification, the read,write and copy test all success.

    Based on the above tests, I think this is a cache consistency issue. Could you provide some infomation about that? We didn't meet such issue on SDK0805, 0902 and 1001 before. 

  • Hi Mingjian,

    I have some slight doubts about the cache coherency, but it is true that J722S has a slightly different architecture for cache than other TDA4 family devices, so will not rule out.

    However, it sounds like a timing issue for pci_epf_test_raise_irq. I think line 600 of pci-epf-test assumes there is enough time for status register to get updated before the IRQ that is raised immediately after gets handled. Perhaps we need a polling while loop that reads the status register on EP side after the write to ensure the write has gone through before raising IRQ.

    Regards,

    Takuma

  • Hi Takuma, 

    In fact, we have already done a read back on the EP side before irq raised. 

    When EP side's prints are always right, RC sometime got the wrong STATUS value. So I think it's  a cache coherency issue. 

    Regards,

    Mingjian

  • Hi Mingjian,

    If reading back the register gets correct value, then it does sound like a cache issue.

    1. Could you try to put memory barriers in place to guarantee write goes through?
    2. Could you share the dmesg logs from the RC-side?
    3. Could you share the lspci -vvv logs from the RC-side?

    Regards,

    Takuma

  • Hi Takuma, 

    >>Could you try to put memory barriers in place to guarantee write goes through?

    Could you provide some modification for example?  

    >>Could you share the dmesg logs from the RC-side?

    root@spartan_a:/# dmesg
    [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
    [    0.000000] Linux version 6.12.35-ti-00958-g952e73839303-dirty (oe-user@oe-host) (aarch64-oe-linux-gcc (GCC) 13.4.0, GNU ld (GNU Binutils) 2.42.0.20240723) #1 SMP PREEMPT Wed Dec 10 08:36:08 UTC 2025
    [    0.000000] KASLR disabled due to lack of seed
    [    0.000000] Machine model: Hirain spartan
    [    0.000000] earlycon: ns16550a0 at MMIO32 0x0000000002800000 (options '')
    [    0.000000] printk: legacy bootconsole [ns16550a0] enabled
    [    0.000000] efi: UEFI not found.
    [    0.000000] OF: reserved mem: 0x000000009e800000..0x000000009fffffff (24576 KiB) nomap non-reusable optee@9e800000
    [    0.000000] OF: reserved mem: 0x0000000080000000..0x000000008007ffff (512 KiB) nomap non-reusable tfa@80000000
    [    0.000000] Reserved memory: created CMA memory pool at 0x00000000f8000000, size 128 MiB
    [    0.000000] OF: reserved mem: initialized node linux-cma-buffers@F8000000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000f8000000..0x00000000ffffffff (131072 KiB) map reusable linux-cma-buffers@F8000000
    [    0.000000] OF: reserved mem: initialized node vision_apps_shared-memories, compatible id dma-heap-carveout
    [    0.000000] OF: reserved mem: 0x00000000ac200000..0x00000000b81fffff (196608 KiB) map non-reusable vision_apps_shared-memories
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000b8200000, size 256 MiB
    [    0.000000] OF: reserved mem: initialized node c7x-ddr-heaps-hi@b8200000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000b8200000..0x00000000c81fffff (262144 KiB) nomap non-reusable c7x-ddr-heaps-hi@b8200000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000aa200000, size 32 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-core-heap-memory-lo@aa200000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000aa200000..0x00000000ac1fffff (32768 KiB) nomap non-reusable vision-apps-core-heap-memory-lo@aa200000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a8300000, size 31 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-c71_1-memory@a8300000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a8300000..0x00000000aa1fffff (31744 KiB) nomap non-reusable vision-apps-c71_1-memory@a8300000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a8200000, size 1 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-c71_1-dma-memory@a8200000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a8200000..0x00000000a82fffff (1024 KiB) nomap non-reusable vision-apps-c71_1-dma-memory@a8200000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a6300000, size 31 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-c71_0-memory@a6300000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a6300000..0x00000000a81fffff (31744 KiB) nomap non-reusable vision-apps-c71_0-memory@a6300000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a6200000, size 1 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-c71-dma-memory@a6200000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a6200000..0x00000000a62fffff (1024 KiB) nomap non-reusable vision-apps-c71-dma-memory@a6200000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a5000000, size 18 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-dma-memory@a5000000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a5000000..0x00000000a61fffff (18432 KiB) nomap non-reusable vision-apps-dma-memory@a5000000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a4000000, size 16 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-rtos-ipc-memory-region@a4000000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a4000000..0x00000000a4ffffff (16384 KiB) nomap non-reusable vision-apps-rtos-ipc-memory-region@a4000000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a2100000, size 31 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-r5f-memory@a2100000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a2100000..0x00000000a3ffffff (31744 KiB) nomap non-reusable vision-apps-r5f-memory@a2100000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a2000000, size 1 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-r5f-dma-memory@a2000000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a2000000..0x00000000a20fffff (1024 KiB) nomap non-reusable vision-apps-r5f-dma-memory@a2000000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a1100000, size 15 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-r5f-memory@a1100000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a1100000..0x00000000a1ffffff (15360 KiB) nomap non-reusable vision-apps-r5f-memory@a1100000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a1000000, size 1 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-r5f-dma-memory@a1000000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a1000000..0x00000000a10fffff (1024 KiB) nomap non-reusable vision-apps-r5f-dma-memory@a1000000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a0100000, size 15 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-r5f-memory@a0100000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a0100000..0x00000000a0ffffff (15360 KiB) nomap non-reusable vision-apps-r5f-memory@a0100000
    [    0.000000] Reserved memory: created DMA memory pool at 0x00000000a0000000, size 1 MiB
    [    0.000000] OF: reserved mem: initialized node vision-apps-r5f-dma-memory@a0000000, compatible id shared-dma-pool
    [    0.000000] OF: reserved mem: 0x00000000a0000000..0x00000000a00fffff (1024 KiB) nomap non-reusable vision-apps-r5f-dma-memory@a0000000
    [    0.000000] Zone ranges:
    [    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
    [    0.000000]   DMA32    empty
    [    0.000000]   Normal   empty
    [    0.000000] Movable zone start for each node
    [    0.000000] Early memory node ranges
    [    0.000000]   node   0: [mem 0x0000000080000000-0x000000008007ffff]
    [    0.000000]   node   0: [mem 0x0000000080080000-0x000000009e7fffff]
    [    0.000000]   node   0: [mem 0x000000009e800000-0x00000000ac1fffff]
    [    0.000000]   node   0: [mem 0x00000000ac200000-0x00000000b81fffff]
    [    0.000000]   node   0: [mem 0x00000000b8200000-0x00000000c81fffff]
    [    0.000000]   node   0: [mem 0x00000000c8200000-0x00000000ffffffff]
    [    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000000ffffffff]
    [    0.000000] psci: probing for conduit method from DT.
    [    0.000000] psci: PSCIv1.1 detected in firmware.
    [    0.000000] psci: Using standard PSCI v0.2 function IDs
    [    0.000000] psci: Trusted OS migration not required
    [    0.000000] psci: SMC Calling Convention v1.5
    [    0.000000] percpu: Embedded 22 pages/cpu s51672 r8192 d30248 u90112
    [    0.000000] pcpu-alloc: s51672 r8192 d30248 u90112 alloc=22*4096
    [    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3
    [    0.000000] Detected VIPT I-cache on CPU0
    [    0.000000] CPU features: detected: GIC system register CPU interface
    [    0.000000] CPU features: detected: ARM erratum 845719
    [    0.000000] alternatives: applying boot alternatives
    [    0.000000] Kernel command line: root=/dev/mmcblk0p13 rootwait ro console=ttyS2,115200n8 earlycon=ns16550a,mmio32,0x02800000 androidboot.soc_name=spartan_a
    [    0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
    [    0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
    [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 524288
    [    0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off
    [    0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 2MB
    [    0.000000] software IO TLB: area num 4.
    [    0.000000] software IO TLB: mapped [mem 0x00000000f5700000-0x00000000f5900000] (2MB)
    [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
    [    0.000000] rcu: Preemptible hierarchical RCU implementation.
    [    0.000000] rcu:     RCU event tracing is enabled.
    [    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=4.
    [    0.000000]  Trampoline variant of Tasks RCU enabled.
    [    0.000000]  Tracing variant of Tasks RCU enabled.
    [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
    [    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
    [    0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.
    [    0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.
    [    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
    [    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
    [    0.000000] GICv3: 256 SPIs implemented
    [    0.000000] GICv3: 0 Extended SPIs implemented
    [    0.000000] Root IRQ handler: gic_handle_irq
    [    0.000000] GICv3: GICv3 features: 16 PPIs
    [    0.000000] GICv3: GICD_CTRL.DS=0, SCR_EL3.FIQ=1
    [    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000001880000
    [    0.000000] ITS [mem 0x01820000-0x0182ffff]
    [    0.000000] GIC: enabling workaround for ITS: Socionext Synquacer pre-ITS
    [    0.000000] ITS@0x0000000001820000: Devices Table too large, reduce ids 20->19
    [    0.000000] ITS@0x0000000001820000: allocated 524288 Devices @80c00000 (flat, esz 8, psz 64K, shr 0)
    [    0.000000] ITS: using cache flushing for cmd queue
    [    0.000000] GICv3: using LPI property table @0x0000000080450000
    [    0.000000] GIC: using cache flushing for LPI property table
    [    0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000080460000
    [    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
    [    0.000000] arch_timer: cp15 timer(s) running at 200.00MHz (phys).
    [    0.000000] clocksource: arch_sys_counter: mask: 0x3ffffffffffffff max_cycles: 0x2e2049d3e8, max_idle_ns: 440795210634 ns
    [    0.000001] sched_clock: 58 bits at 200MHz, resolution 5ns, wraps every 4398046511102ns
    [    0.008805] Console: colour dummy device 80x25
    [    0.013396] Calibrating delay loop (skipped), value calculated using timer frequency.. 400.00 BogoMIPS (lpj=800000)
    [    0.024066] pid_max: default: 32768 minimum: 301
    [    0.028866] LSM: initializing lsm=capability,selinux
    [    0.033990] SELinux:  Initializing.
    [    0.037981] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
    [    0.045546] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
    [    0.055552] rcu: Hierarchical SRCU implementation.
    [    0.060496] rcu:     Max phase no-delay instances is 1000.
    [    0.066015] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level
    [    0.074861] EFI services will not be available.
    [    0.079756] smp: Bringing up secondary CPUs ...
    [    0.084965] Detected VIPT I-cache on CPU1
    [    0.085042] GICv3: CPU1: found redistributor 1 region 0:0x00000000018a0000
    [    0.085059] GICv3: CPU1: using allocated LPI pending table @0x0000000080470000
    [    0.085103] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
    [    0.085806] Detected VIPT I-cache on CPU2
    [    0.085864] GICv3: CPU2: found redistributor 2 region 0:0x00000000018c0000
    [    0.085878] GICv3: CPU2: using allocated LPI pending table @0x0000000080480000
    [    0.085907] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
    [    0.086536] Detected VIPT I-cache on CPU3
    [    0.086593] GICv3: CPU3: found redistributor 3 region 0:0x00000000018e0000
    [    0.086607] GICv3: CPU3: using allocated LPI pending table @0x0000000080490000
    [    0.086634] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
    [    0.086752] smp: Brought up 1 node, 4 CPUs
    [    0.166383] SMP: Total of 4 processors activated.
    [    0.171189] CPU: All CPU(s) started at EL2
    [    0.175387] CPU features: detected: 32-bit EL0 Support
    [    0.180642] CPU features: detected: CRC32 instructions
    [    0.185933] alternatives: applying system-wide alternatives
    [    0.192215] Memory: 1209728K/2097152K available (13184K kernel code, 1270K rwdata, 4728K rodata, 2752K init, 888K bss, 747464K reserved, 131072K cma-reserved)
    [    0.207701] devtmpfs: initialized
    [    0.222171] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
    [    0.232172] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
    [    0.242575] 27008 pages in range for non-PLT usage
    [    0.242598] 518528 pages in range for PLT usage
    [    0.247790] pinctrl core: initialized pinctrl subsystem
    [    0.258309] DMI not present or invalid.
    [    0.264344] NET: Registered PF_NETLINK/PF_ROUTE protocol family
    [    0.271530] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations
    [    0.279335] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
    [    0.288001] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
    [    0.296157] audit: initializing netlink subsys (disabled)
    [    0.301928] audit: type=2000 audit(0.184:1): state=initialized audit_enabled=0 res=1
    [    0.302380] thermal_sys: Registered thermal governor 'step_wise'
    [    0.309854] thermal_sys: Registered thermal governor 'power_allocator'
    [    0.316035] cpuidle: using governor menu
    [    0.326867] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
    [    0.333891] ASID allocator initialised with 65536 entries
    [    0.345979] /bus@f0000/interrupt-controller@1800000: Fixed dependency cycle(s) with /bus@f0000/interrupt-controller@1800000
    [    0.370566] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
    [    0.377532] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page
    [    0.383935] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages
    [    0.390874] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page
    [    0.397277] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
    [    0.404209] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page
    [    0.410610] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages
    [    0.417541] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page
    [    0.425594] k3-chipinfo 43000014.chipid: Family:J722S rev:SR1.0 JTAGID[0x0bba002f] Detected
    [    0.434861] iommu: Default domain type: Translated
    [    0.439788] iommu: DMA domain TLB invalidation policy: strict mode
    [    0.446876] SCSI subsystem initialized
    [    0.450938] libata version 3.00 loaded.
    [    0.451133] usbcore: registered new interface driver usbfs
    [    0.456773] usbcore: registered new interface driver hub
    [    0.462228] usbcore: registered new device driver usb
    [    0.467957] pps_core: LinuxPPS API ver. 1 registered
    [    0.473036] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
    [    0.482377] PTP clock support registered
    [    0.486413] EDAC MC: Ver: 3.0.0
    [    0.489942] scmi_core: SCMI protocol bus registered
    [    0.495213] FPGA manager framework
    [    0.498761] Advanced Linux Sound Architecture Driver Initialized.
    [    0.505904] vgaarb: loaded
    [    0.509038] clocksource: Switched to clocksource arch_sys_counter
    [    0.515516] VFS: Disk quotas dquot_6.6.0
    [    0.519545] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
    [    0.533646] Carveout Heap: Exported 192 MiB at 0x00000000ac200000
    [    0.539974] NET: Registered PF_INET protocol family
    [    0.545106] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)
    [    0.554275] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)
    [    0.563058] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
    [    0.570991] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
    [    0.579183] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)
    [    0.587041] TCP: Hash tables configured (established 16384 bind 16384)
    [    0.593862] UDP hash table entries: 1024 (order: 3, 32768 bytes, linear)
    [    0.600768] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes, linear)
    [    0.608238] NET: Registered PF_UNIX/PF_LOCAL protocol family
    [    0.614479] RPC: Registered named UNIX socket transport module.
    [    0.620549] RPC: Registered udp transport module.
    [    0.625361] RPC: Registered tcp transport module.
    [    0.630165] RPC: Registered tcp-with-tls transport module.
    [    0.635770] RPC: Registered tcp NFSv4.1 backchannel transport module.
    [    0.642353] NET: Registered PF_XDP protocol family
    [    0.647262] PCI: CLS 0 bytes, default 64
    [    0.652560] Initialise system trusted keyrings
    [    0.657330] workingset: timestamp_bits=46 max_order=19 bucket_order=0
    [    0.664213] squashfs: version 4.0 (2009/01/31) Phillip Lougher
    [    0.670442] NFS: Registering the id_resolver key type
    [    0.675635] Key type id_resolver registered
    [    0.679909] Key type id_legacy registered
    [    0.684022] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
    [    0.690879] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
    [    0.751309] Key type asymmetric registered
    [    0.755505] Asymmetric key parser 'x509' registered
    [    0.760545] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 244)
    [    0.768291] io scheduler mq-deadline registered
    [    0.772925] io scheduler kyber registered
    [    0.777055] io scheduler bfq registered
    [    0.784250] pinctrl-single 4084000.pinctrl: 34 pins, size 136
    [    0.790896] pinctrl-single f4000.pinctrl: 171 pins, size 684
    [    0.798115] gpio_func hb1-out1-en: GPIO requested is not here yet, deferring the probe
    [    0.806303] gpio_func hb1-out2-en: GPIO requested is not here yet, deferring the probe
    [    0.814470] gpio_func hb2-out1-en: GPIO requested is not here yet, deferring the probe
    [    0.822616] gpio_func hb2-out2-en: GPIO requested is not here yet, deferring the probe
    [    0.830761] gpio_func can-vcc: GPIO requested is not here yet, deferring the probe
    [    0.838930] ledtrig-cpu: registered to indicate activity on CPUs
    [    0.851435] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
    [    0.866615] loop: module loaded
    [    0.870951] megasas: 07.727.03.00-rc1
    [    0.879054] tun: Universal TUN/TAP device driver, 1.6
    [    0.885453] VFIO - User Level meta-driver version: 0.3
    [    0.892198] usbcore: registered new interface driver usb-storage
    [    0.899053] i2c_dev: i2c /dev entries driver
    [    0.904620] sdhci: Secure Digital Host Controller Interface driver
    [    0.910949] sdhci: Copyright(c) Pierre Ossman
    [    0.915661] sdhci-pltfm: SDHCI platform and OF driver helper
    [    0.922234] SMCCC: SOC_ID: ARCH_SOC_ID not implemented, skipping ....
    [    0.929722] usbcore: registered new interface driver usbhid
    [    0.935425] usbhid: USB HID core driver
    [    0.940765] hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 (0,8000003f) counters available
    [    0.951267] optee: probing for conduit method.
    [    0.955846] optee: revision 4.6 (71785645fa6ce42d)
    [    0.956192] optee: dynamic shared memory is enabled
    [    0.966756] optee: initialized driver
    [    0.966864] random: crng init done
    [    0.972678] Initializing XFRM netlink socket
    [    0.978424] NET: Registered PF_PACKET protocol family
    [    0.983662] Key type dns_resolver registered
    [    0.996559] registered taskstats version 1
    [    1.001051] Loading compiled-in X.509 certificates
    [    1.015951] gpio_func hb1-out1-en: GPIO requested is not here yet, deferring the probe
    [    1.024280] gpio_func hb1-out2-en: GPIO requested is not here yet, deferring the probe
    [    1.032694] gpio_func hb2-out1-en: GPIO requested is not here yet, deferring the probe
    [    1.041012] gpio_func hb2-out2-en: GPIO requested is not here yet, deferring the probe
    [    1.049420] gpio_func can-vcc: GPIO requested is not here yet, deferring the probe
    [    1.058858] ti-sci 44043000.system-controller: ABI: 4.0 (firmware rev 0x000b '11.1.9-v11.01.09 (Fancy Rat)')
    [    1.114529] omap_i2c 4900000.i2c: bus 0 rev0.12 at 400 kHz
    [    1.121330] omap_i2c 2b200000.i2c: bus 1 rev0.12 at 400 kHz
    [    1.127796] ti-sci-intr 4210000.interrupt-controller: Interrupt Router 5 domain created
    [    1.136137] ti-sci-intr bus@f0000:interrupt-controller@a00000: Interrupt Router 3 domain created
    [    1.145357] ti-sci-inta 48000000.interrupt-controller: Interrupt Aggregator domain 28 created
    [    1.154333] ti-sci-inta 4e400000.interrupt-controller: Interrupt Aggregator domain 200 created
    [    1.167857] gpio_func hb1-out1-en: GPIO requested is not here yet, deferring the probe
    [    1.176333] gpio_func hb1-out2-en: GPIO requested is not here yet, deferring the probe
    [    1.184647] gpio_func hb2-out1-en: GPIO requested is not here yet, deferring the probe
    [    1.192921] gpio_func hb2-out2-en: GPIO requested is not here yet, deferring the probe
    [    1.201212] gpio_func can-vcc: GPIO requested is not here yet, deferring the probe
    [    1.209864] ti-udma 485c0100.dma-controller: Number of rings: 82
    [    1.218321] ti-udma 485c0100.dma-controller: Channels: 44 (bchan: 16, tchan: 12, rchan: 16)
    [    1.229120] ti-udma 485c0000.dma-controller: Number of rings: 150
    [    1.239918] ti-udma 485c0000.dma-controller: Channels: 35 (tchan: 20, rchan: 15)
    [    1.249449] ti-udma 4e230000.dma-controller: Number of rings: 40
    [    1.257120] ti-udma 4e230000.dma-controller: Channels: 20 (bchan: 0, tchan: 8, rchan: 12)
    [    1.267318] printk: legacy console [ttyS2] disabled
    [    1.272688] 2800000.serial: ttyS2 at MMIO 0x2800000 (irq = 267, base_baud = 3000000) is a 8250
    [    1.281559] printk: legacy console [ttyS2] enabled
    [    1.291264] printk: legacy bootconsole [ns16550a0] disabled
    [    1.307335] spi-nor spi0.0: unrecognized JEDEC id bytes: 00 00 00 00 00 00
    [    1.314287] spi spi0.0: chipselect 0 already in use
    [    1.319164] spi_master spi0: spi_device register error /bus@f0000/bus@fc00000/spi@fc40000/nand@0
    [    1.327953] spi_master spi0: Failed to create SPI device for /bus@f0000/bus@fc00000/spi@fc40000/nand@0
    [    1.337748] am65-cpsw-nuss 8000000.ethernet: initializing am65 cpsw nuss version 0x6BA01903, cpsw version 0x6BA81903 Ports: 3 quirks:00000006
    [    1.350776] am65-cpsw-nuss 8000000.ethernet: Port 2: fixed-link
    [    1.361951] mmc0: CQHCI version 5.10
    [    1.401716] mmc0: SDHCI controller on fa10000.mmc [fa10000.mmc] using ADMA 64-bit
    [    1.437420] vcc0v85_ram_buck3: Bringing 750000uV into 850000-850000uV
    [    1.455682] gpio_func hb1-out1-en: 575: GPIO pin number
    [    1.460961] GPIO pin number is 575
    [    1.464444] gpio_func hb1-out1-en: gpio_func_probe: probe success!
    [    1.470784] gpio_func hb1-out2-en: 570: GPIO pin number
    [    1.476029] GPIO pin number is 570
    [    1.479572] mmc0: Command Queue Engine enabled
    [    1.479602] gpio_func hb1-out2-en: gpio_func_probe: probe success!
    [    1.484032] mmc0: new HS200 MMC card at address 0001
    [    1.490412] gpio_func hb2-out1-en: 569: GPIO pin number
    [    1.495754] mmcblk0: mmc0:0001 8GUF4R 7.28 GiB
    [    1.500386] GPIO pin number is 569
    [    1.506650] Alternate GPT is invalid, using primary GPT.
    [    1.508320] gpio_func hb2-out1-en: gpio_func_probe: probe success!
    [    1.513580]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16 p17 p18
    [    1.519901] gpio_func hb2-out2-en: 574: GPIO pin number
    [    1.529618] mmcblk0boot0: mmc0:0001 8GUF4R 31.9 MiB
    [    1.532658] GPIO pin number is 574
    [    1.538443] mmcblk0boot1: mmc0:0001 8GUF4R 31.9 MiB
    [    1.540997] gpio_func hb2-out2-en: gpio_func_probe: probe success!
    [    1.546621] mmcblk0rpmb: mmc0:0001 8GUF4R 4.00 MiB, chardev (238:0)
    [    1.552184] gpio_func can-vcc: 602: GPIO pin number
    [    1.563125] GPIO pin number is 602
    [    1.566615] gpio_func can-vcc: gpio_func_probe: probe success!
    [    1.572997] j721e-pcie f102000.pcie: host bridge /pcie@f102000 ranges:
    [    1.579565] j721e-pcie f102000.pcie:       IO 0x0600001000..0x0600100fff -> 0x0000001000
    [    1.587673] j721e-pcie f102000.pcie:      MEM 0x0600101000..0x06ffffffff -> 0x0000101000
    [    1.595769] j721e-pcie f102000.pcie:   IB MEM 0x0000000000..0xffffffffffff -> 0x0000000000
    [    2.607758] j721e-pcie f102000.pcie: PCI host bridge to bus 0000:00
    [    2.614090] pci_bus 0000:00: root bus resource [bus 00-ff]
    [    2.619577] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0x1000-0x100fff])
    [    2.628614] pci_bus 0000:00: root bus resource [mem 0x600101000-0x6ffffffff] (bus address [0x00101000-0xffffffff])
    [    2.638978] pci 0000:00:00.0: [104c:b010] type 01 class 0x060400 PCIe Root Port
    [    2.646293] pci 0000:00:00.0: PCI bridge to [bus 00]
    [    2.651258] pci 0000:00:00.0:   bridge window [io  0x0000-0x0fff]
    [    2.657346] pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
    [    2.664130] pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
    [    2.671929] pci 0000:00:00.0: supports D1
    [    2.675936] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
    [    2.684669] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
    [    2.692875] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
    [    2.699512] pci 0000:00:00.0: PCI bridge to [bus 01]
    [    2.704480] pci_bus 0000:00: resource 4 [io  0x0000-0xfffff]
    [    2.710135] pci_bus 0000:00: resource 5 [mem 0x600101000-0x6ffffffff]
    [    2.716677] pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
    [    2.723615] pcieport 0000:00:00.0: PME: Signaling with IRQ 513
    [    2.729750] pcieport 0000:00:00.0: AER: enabled with IRQ 513
    [    2.736116] am65-cpsw-nuss 8000000.ethernet: initializing am65 cpsw nuss version 0x6BA01903, cpsw version 0x6BA81903 Ports: 3 quirks:00000006
    [    2.749015] am65-cpsw-nuss 8000000.ethernet: Port 2: fixed-link
    [    2.756170] am65-cpsw-nuss 8000000.ethernet: Use random MAC address
    [    2.763063] davinci_mdio 8000f00.mdio: Configuring MDIO in manual mode
    [    2.805053] davinci_mdio 8000f00.mdio: davinci mdio revision 17.7, bus freq 1000000
    [    2.825882] NXP C45 TJA1103 8000f00.mdio:01: the phy does not support MACsec
    [    2.834465] davinci_mdio 8000f00.mdio: phy[1]: device 8000f00.mdio:01, driver NXP C45 TJA1103
    [    2.843282] am65-cpsw-nuss 8000000.ethernet: initialized cpsw ale version 1.5
    [    2.850421] am65-cpsw-nuss 8000000.ethernet: ALE Table size 512, Policers 32
    [    2.858271] am65-cpsw-nuss 8000000.ethernet: CPTS ver 0x4e8a010d, freq:500000000, add_val:1 pps:1
    [    2.876812] am65-cpsw-nuss 8000000.ethernet: set new flow-id-base 19
    [    2.885022] am65-cpsw-nuss 8000000.ethernet: Port 0: Using custom interface name 'etha1'
    [    2.893920] am65-cpsw-nuss 8000000.ethernet: Port 1: Using custom interface name 'etha2'
    [    2.905093] clk: Disabling unused clocks
    [    2.914189] PM: genpd: Disabling unused power domains
    [    2.919340] ALSA device list:
    [    2.922381]   No soundcards found.
    [    2.936609] EXT4-fs (mmcblk0p13): orphan cleanup on readonly fs
    [    2.942899] EXT4-fs (mmcblk0p13): mounted filesystem ba04aef1-e9c9-4a49-b14f-8f38b82c6755 ro with ordered data mode. Quota mode: none.
    [    2.955055] VFS: Mounted root (ext4 filesystem) readonly on device 179:13.
    [    2.962241] devtmpfs: mounted
    [    2.966007] Freeing unused kernel memory: 2752K
    [    2.970634] Run /sbin/init as init process
    [    2.974729]   with arguments:
    [    2.974733]     /sbin/init
    [    2.974737]   with environment:
    [    2.974741]     HOME=/
    [    2.974745]     TERM=linux
    [    3.067184] systemd[1]: System time before build time, advancing clock.
    [    3.093949] systemd[1]: systemd 255.21^ running in system mode (+PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK +SECCOMP -GCRYPT -GNUTLS -OPENSSL +ACL +BLKID -CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -TPM2 -BZIP2 -LZ4 -XZ -ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON -UTMP +SYSVINIT default-hierarchy=unified)
    [    3.125846] systemd[1]: Detected architecture arm64.
    [    3.141540] systemd[1]: Hostname set to <spartan>.
    [    3.148281] systemd[1]: Initializing machine ID from random generator.
    [    3.155038] systemd[1]: Installed transient /etc/machine-id file.
    [    3.554372] systemd[1]: Queued start job for default target Multi-User System.
    [    3.609284] systemd[1]: Created slice Slice /system/getty.
    [    3.626879] systemd[1]: Created slice Slice /system/modprobe.
    [    3.646933] systemd[1]: Created slice Slice /system/serial-getty.
    [    3.666879] systemd[1]: Created slice Slice /system/systemd-fsck.
    [    3.686903] systemd[1]: Created slice Slice /system/systemd-makefs.
    [    3.706408] systemd[1]: Created slice User and Session Slice.
    [    3.725505] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
    [    3.745386] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
    [    3.765240] systemd[1]: Expecting device /dev/disk/by-partlabel/data...
    [    3.785138] systemd[1]: Expecting device /dev/disk/by-partlabel/update...
    [    3.805261] systemd[1]: Reached target Path Units.
    [    3.821187] systemd[1]: Reached target Remote File Systems.
    [    3.837180] systemd[1]: Reached target Slice Units.
    [    3.849206] systemd[1]: Reached target Swaps.
    [    3.861935] systemd[1]: Listening on Syslog Socket.
    [    3.874322] systemd[1]: Listening on Journal Audit Socket.
    [    3.893752] systemd[1]: Listening on Journal Socket (/dev/log).
    [    3.913871] systemd[1]: Listening on Journal Socket.
    [    3.929992] systemd[1]: Listening on Network Service Netlink Socket.
    [    3.950332] systemd[1]: Listening on udev Control Socket.
    [    3.969716] systemd[1]: Listening on udev Kernel Socket.
    [    3.985791] systemd[1]: Listening on User Database Manager Socket.
    [    4.033433] systemd[1]: Mounting Huge Pages File System...
    [    4.048785] systemd[1]: Mounting POSIX Message Queue File System...
    [    4.072959] systemd[1]: Mounting Kernel Debug File System...
    [    4.085783] systemd[1]: Kernel Trace File System was skipped because of an unmet condition check (ConditionPathExists=/sys/kernel/tracing).
    [    4.102729] systemd[1]: Mounting Temporary Directory /tmp...
    [    4.121600] systemd[1]: Create List of Static Device Nodes was skipped because of an unmet condition check (ConditionFileNotEmpty=/lib/modules/6.12.35-ti-00958-g952e73839303-dirty/modules.devname).
    [    4.143703] systemd[1]: Starting Load Kernel Module configfs...
    [    4.161681] systemd[1]: Starting Load Kernel Module drm...
    [    4.177813] systemd[1]: Starting Load Kernel Module fuse...
    [    4.197338] systemd[1]: Starting Journal Service...
    [    4.210784] systemd[1]: Load Kernel Modules was skipped because no trigger condition checks were met.
    [    4.224104] systemd[1]: Starting Generate network units from Kernel command line...
    [    4.249682] systemd[1]: Starting Remount Root and Kernel File Systems...
    [    4.273785] systemd[1]: Starting Apply Kernel Variables...
    [    4.294060] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...
    [    4.306208] systemd-journald[111]: Collecting audit messages is enabled.
    [    4.325633] EXT4-fs (mmcblk0p13): re-mounted ba04aef1-e9c9-4a49-b14f-8f38b82c6755.
    [    4.330304] systemd[1]: Starting Coldplug All udev Devices...
    [    4.364778] systemd[1]: Mounted Huge Pages File System.
    [    4.386061] systemd[1]: Mounted POSIX Message Queue File System.
    [    4.410125] systemd[1]: Mounted Kernel Debug File System.
    [    4.433701] systemd[1]: Started Journal Service.
    [    4.692706] systemd-journald[111]: Received client request to flush runtime journal.
    [    4.835472] audit: type=1334 audit(1640966401.764:2): prog-id=6 op=LOAD
    [    4.842867] audit: type=1334 audit(1640966401.772:3): prog-id=7 op=LOAD
    [    6.287640] EXT4-fs (mmcblk0p16): mounted filesystem efab1203-11a3-437a-bfe0-79f686f7d21d r/w with ordered data mode. Quota mode: none.
    [    6.379457] EXT4-fs (mmcblk0p18): mounted filesystem ba2e0cc8-6517-40da-9536-0f227f897e38 r/w with ordered data mode. Quota mode: none.
    [    6.570178] audit: type=1334 audit(1640966403.500:4): prog-id=8 op=LOAD
    [    6.608228] audit: type=1334 audit(1640966403.536:5): prog-id=9 op=LOAD
    [    6.614911] audit: type=1334 audit(1640966403.544:6): prog-id=10 op=LOAD
    [    6.621662] audit: type=1334 audit(1640966403.552:7): prog-id=11 op=LOAD
    [    7.233331] audit: type=1334 audit(1640966404.160:8): prog-id=12 op=LOAD
    [    7.240190] audit: type=1334 audit(1640966404.168:9): prog-id=13 op=LOAD
    [    7.246979] audit: type=1334 audit(1640966404.176:10): prog-id=14 op=LOAD
    [    7.463411] audit: type=1334 audit(1640966404.392:11): prog-id=15 op=LOAD
    [    7.632494] cgroup: Unknown subsys name 'memory'
    [    7.706670] file system registered
    [    7.779994] am65-cpsw-nuss 8000000.ethernet etha1: PHY [8000f00.mdio:01] driver [NXP C45 TJA1103] (irq=POLL)
    [    7.790098] audit: type=1334 audit(1640966404.720:12): prog-id=16 op=LOAD
    [    7.790621] am65-cpsw-nuss 8000000.ethernet etha1: configuring for phy/rgmii-txid link mode
    [    7.825520] am65-cpsw-nuss 8000000.ethernet etha2: configuring for fixed/sgmii link mode
    [    7.834195] am65-cpsw-nuss 8000000.ethernet etha2: Link is Up - 1Gbps/Full - flow control off
    [    8.008802] ffs_data_put(): freeing
    [    8.144560] read descriptors
    [    8.147748] bcdVersion must be 0x0100, stored in Little Endian order. Userspace driver should be fixed, accepting 0x0001 for compatibility.
    [    8.161148] read strings
    [   18.341994] platform fa00000.mmc: deferred probe pending: platform: supplier regulator-mmc1 not ready
    [  948.472787] pci 0000:01:00.0: [104c:b010] type 00 class 0xff0000 PCIe Endpoint
    [  948.480110] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x000000ff]
    [  948.486068] pci 0000:01:00.0: BAR 1 [mem 0x00000000-0x000001ff]
    [  948.492013] pci 0000:01:00.0: BAR 2 [mem 0x00000000-0x000003ff]
    [  948.497954] pci 0000:01:00.0: BAR 3 [mem 0x00000000-0x00003fff]
    [  948.503895] pci 0000:01:00.0: BAR 4 [mem 0x00000000-0x0001ffff]
    [  948.509832] pci 0000:01:00.0: BAR 5 [mem 0x00000000-0x000fffff]
    [  948.516014] pci 0000:01:00.0: supports D1
    [  948.520036] pci 0000:01:00.0: PME# supported from D0 D1 D3hot
    [  948.533230] pcieport 0000:00:00.0: bridge window [mem 0x600200000-0x6003fffff]: assigned
    [  948.541400] pci 0000:01:00.0: BAR 5 [mem 0x600200000-0x6002fffff]: assigned
    [  948.548394] pci 0000:01:00.0: BAR 4 [mem 0x600300000-0x60031ffff]: assigned
    [  948.555391] pci 0000:01:00.0: BAR 3 [mem 0x600320000-0x600323fff]: assigned
    [  948.562371] pci 0000:01:00.0: BAR 2 [mem 0x600324000-0x6003243ff]: assigned
    [  948.569360] pci 0000:01:00.0: BAR 1 [mem 0x600324400-0x6003245ff]: assigned
    [  948.576338] pci 0000:01:00.0: BAR 0 [mem 0x600324600-0x6003246ff]: assigned
    root@spartan_a:/#

    >>Could you share the lspci -vvv logs from the RC-side?

    root@spartan_a:/# lspci
    00:00.0 PCI bridge: Texas Instruments Device b010
    01:00.0 Unassigned class [ff00]: Texas Instruments Device b010
    root@spartan_a:/# lspci -vvv
    00:00.0 PCI bridge: Texas Instruments Device b010 (prog-if 00 [Normal decode])
            Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
            Latency: 0
            Interrupt: pin A routed to IRQ 513
            Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
            I/O behind bridge: 0000f000-00000fff [disabled] [32-bit]
            Memory behind bridge: 00200000-003fffff [size=2M] [32-bit]
            Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff [disabled] [64-bit]
            Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
            BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
                    PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
            Capabilities: [80] Power Management version 3
                    Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
                    Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
            Capabilities: [90] MSI: Enable+ Count=1/1 Maskable+ 64bit+
                    Address: 0000000001000000  Data: 0000
                    Masking: 00000000  Pending: 00000000
            Capabilities: [b0] MSI-X: Enable- Count=1 Masked-
                    Vector table: BAR=0 offset=00000000
                    PBA: BAR=0 offset=00000008
            Capabilities: [c0] Express (v2) Root Port (Slot+), IntMsgNum 0
                    DevCap: MaxPayload 128 bytes, PhantFunc 0
                            ExtTag- RBE+
                    DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
                            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                            MaxPayload 128 bytes, MaxReadReq 512 bytes
                    DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
                    LnkCap: Port #0, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <8us
                            ClockPM- Surprise- LLActRep- BwNot+ ASPMOptComp+
                    LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk-
                            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                    LnkSta: Speed 8GT/s, Width x1
                            TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt+
                    SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                            Slot #0, PowerLimit 0W; Interlock- NoCompl-
                    SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                            Control: AttnInd Off, PwrInd Off, Power+ Interlock-
                    SltSta: Status: AttnBtn- PowerFlt- MRL+ CmdCplt- PresDet- Interlock-
                            Changed: MRL- PresDet- LinkState-
                    RootCap: CRSVisible-
                    RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
                    RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                    DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR+
                             10BitTagComp- 10BitTagReq- OBFF Via message, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
                             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                             FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd-
                             AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS-
                    DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd-
                             AtomicOpsCtl: ReqEn- EgressBlck-
                             IDOReq- IDOCompl- LTR+ EmergencyPowerReductionReq-
                             10BitTagReq- OBFF Disabled, EETLPPrefixBlk-
                    LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
                    LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
                             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                             Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
                    LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+ EqualizationPhase1+
                             EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
                             Retimer- 2Retimers- CrosslinkRes: unsupported
            Capabilities: [100 v2] Advanced Error Reporting
                    UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                    UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                    UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                    CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
                    CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
                    AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
                            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
                    HeaderLog: 00000000 00000000 00000000 00000000
                    RootCmd: CERptEn+ NFERptEn+ FERptEn+
                    RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
                             FirstFatal- NonFatalMsg- FatalMsg- IntMsgNum 0
                    ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
            Capabilities: [150 v1] Device Serial Number 00-00-00-00-00-00-00-00
            Capabilities: [300 v1] Secondary PCI Express
                    LnkCtl3: LnkEquIntrruptEn- PerformEqu-
                    LaneErrStat: 0
            Capabilities: [4c0 v1] Virtual Channel
                    Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                    Arb:    Fixed- WRR32- WRR64- WRR128-
                    Ctrl:   ArbSelect=Fixed
                    Status: InProgress-
                    VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                            Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
                            Status: NegoPending- InProgress-
                    VC1:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                            Ctrl:   Enable- ID=1 ArbSelect=Fixed TC/VC=00
                            Status: NegoPending- InProgress-
                    VC2:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                            Ctrl:   Enable- ID=2 ArbSelect=Fixed TC/VC=00
                            Status: NegoPending- InProgress-
                    VC3:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                            Ctrl:   Enable- ID=3 ArbSelect=Fixed TC/VC=00
                            Status: NegoPending- InProgress-
            Capabilities: [900 v1] L1 PM Substates
                    L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
                              PortCommonModeRestoreTime=255us PortTPowerOnTime=26us
                    L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
                               T_CommonMode=255us LTR1.2_Threshold=287744ns
                    L1SubCtl2: T_PwrOn=26us
            Capabilities: [a20 v1] Precision Time Measurement
                    PTMCap: Requester- Responder+ Root+
                    PTMClockGranularity: 4ns
                    PTMControl: Enabled- RootSelected-
                    PTMEffectiveGranularity: Unknown
            Kernel driver in use: pcieport
    lspci: Unable to load libkmod resources: error -2
    
    01:00.0 Unassigned class [ff00]: Texas Instruments Device b010
            Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
            Interrupt: pin A routed to IRQ 255
            Region 0: Memory at 600324600 (32-bit, non-prefetchable) [disabled] [size=256]
            Region 1: Memory at 600324400 (32-bit, non-prefetchable) [disabled] [size=512]
            Region 2: Memory at 600324000 (32-bit, non-prefetchable) [disabled] [size=1K]
            Region 3: Memory at 600320000 (32-bit, non-prefetchable) [disabled] [size=16K]
            Region 4: Memory at 600300000 (32-bit, non-prefetchable) [disabled] [size=128K]
            Region 5: Memory at 600200000 (32-bit, non-prefetchable) [disabled] [size=1M]
            Capabilities: [80] Power Management version 3
                    Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
                    Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
            Capabilities: [90] MSI: Enable- Count=1/2 Maskable- 64bit+
                    Address: 0000000000000000  Data: 0000
            Capabilities: [b0] MSI-X: Enable- Count=2 Masked-
                    Vector table: BAR=0 offset=00000080
                    PBA: BAR=0 offset=000000a0
            Capabilities: [c0] Express (v2) Endpoint, IntMsgNum 0
                    DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <1us, L1 <1us
                            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W
                    DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
                            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
                            MaxPayload 128 bytes, MaxReadReq 512 bytes
                    DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
                    LnkCap: Port #0, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <8us
                            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                    LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk-
                            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                    LnkSta: Speed 8GT/s, Width x1
                            TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
                    DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR+
                             10BitTagComp- 10BitTagReq- OBFF Via message, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
                             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                             FRS- TPHComp- ExtTPHComp-
                             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                    DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
                             AtomicOpsCtl: ReqEn-
                             IDOReq- IDOCompl- LTR+ EmergencyPowerReductionReq-
                             10BitTagReq- OBFF Disabled, EETLPPrefixBlk-
                    LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
                    LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
                             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                             Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
                    LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+ EqualizationPhase1+
                             EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
                             Retimer- 2Retimers- CrosslinkRes: unsupported
            Capabilities: [100 v2] Advanced Error Reporting
                    UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                    UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                    UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                    CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
                    CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
                    AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
                            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
                    HeaderLog: 00000000 00000000 00000000 00000000
            Capabilities: [150 v1] Device Serial Number 00-00-00-00-00-00-00-00
            Capabilities: [160 v1] Power Budgeting <?>
            Capabilities: [1b8 v1] Latency Tolerance Reporting
                    Max snoop latency: 0ns
                    Max no snoop latency: 0ns
            Capabilities: [1c0 v1] Dynamic Power Allocation <?>
            Capabilities: [300 v1] Secondary PCI Express
                    LnkCtl3: LnkEquIntrruptEn- PerformEqu-
                    LaneErrStat: 0
            Capabilities: [400 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
            Capabilities: [440 v1] Process Address Space ID (PASID)
                    PASIDCap: Exec+ Priv+, Max PASID Width: 14
                    PASIDCtl: Enable- Exec- Priv-
            Capabilities: [4c0 v1] Virtual Channel
                    Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                    Arb:    Fixed- WRR32- WRR64- WRR128-
                    Ctrl:   ArbSelect=Fixed
                    Status: InProgress-
                    VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                            Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
                            Status: NegoPending- InProgress-
                    VC1:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                            Ctrl:   Enable- ID=1 ArbSelect=Fixed TC/VC=00
                            Status: NegoPending- InProgress-
                    VC2:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                            Ctrl:   Enable- ID=2 ArbSelect=Fixed TC/VC=00
                            Status: NegoPending- InProgress-
                    VC3:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                            Ctrl:   Enable- ID=3 ArbSelect=Fixed TC/VC=00
                            Status: NegoPending- InProgress-
            Capabilities: [900 v1] L1 PM Substates
                    L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
                              PortCommonModeRestoreTime=255us PortTPowerOnTime=26us
                    L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
                               T_CommonMode=0us LTR1.2_Threshold=287744ns
                    L1SubCtl2: T_PwrOn=26us
            Capabilities: [a20 v1] Precision Time Measurement
                    PTMCap: Requester+ Responder- Root-
                    PTMClockGranularity: Unimplemented
                    PTMControl: Enabled- RootSelected-
                    PTMEffectiveGranularity: Unknown
    
    root@spartan_a:/#

    Regards,

    Mingjian

  • Hi Mingjian,

    Looks like all of the memory regions are non-prefetchable, so it should not be doing prefetching:

    Region 0: Memory at 600324600 (32-bit, non-prefetchable) [disabled] [size=256]
    Region 1: Memory at 600324400 (32-bit, non-prefetchable) [disabled] [size=512]
    Region 2: Memory at 600324000 (32-bit, non-prefetchable) [disabled] [size=1K]
    Region 3: Memory at 600320000 (32-bit, non-prefetchable) [disabled] [size=16K]
    Region 4: Memory at 600300000 (32-bit, non-prefetchable) [disabled] [size=128K]
    Region 5: Memory at 600200000 (32-bit, non-prefetchable) [disabled] [size=1M]

    On the other hand, I notice that the PCIe related boot logs occur later in the boot:

    [ 948.472787] pci 0000:01:00.0: [104c:b010] type 00 class 0xff0000 PCIe Endpoint
    [ 948.480110] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x000000ff]
    [ 948.486068] pci 0000:01:00.0: BAR 1 [mem 0x00000000-0x000001ff]
    [ 948.492013] pci 0000:01:00.0: BAR 2 [mem 0x00000000-0x000003ff]
    [ 948.497954] pci 0000:01:00.0: BAR 3 [mem 0x00000000-0x00003fff]
    [ 948.503895] pci 0000:01:00.0: BAR 4 [mem 0x00000000-0x0001ffff]
    [ 948.509832] pci 0000:01:00.0: BAR 5 [mem 0x00000000-0x000fffff]
    [ 948.516014] pci 0000:01:00.0: supports D1
    [ 948.520036] pci 0000:01:00.0: PME# supported from D0 D1 D3hot

    Regards,

    Takuma

  • To add on to previous post,

    • What is the procedure when enabling EP and RC? That is, what commands are ran in what order, on which board?
    • As for the memory barrier, you could use smp_mb() or other memory barrier related API.

    Regards,

    Takuma

  • Hi Takuma, 

    The command we use: 

    EP first:

    insmod pci-epf-test.ko

    mount -t configfs none /sys/kernel/config

    cd /sys/kernel/config/pci_ep/
    mkdir functions/pci_epf_test/func1
    echo 0x104c > functions/pci_epf_test/func1/vendorid
    echo 0xb010 > functions/pci_epf_test/func1/deviceid
    echo 2 > functions/pci_epf_test/func1/msi_interrupts
    echo 2 > functions/pci_epf_test/func1/msix_interrupts
    ln -s functions/pci_epf_test/func1 controllers/f102000.pcie-ep/


    echo 1 > controllers/f102000.pcie-ep/start

    Then RC:

    insmod pci_endpoint_test.ko

    echo 1 > /sys/bus/pci/rescan 

    Regards, 

    Mingjian

  • Hi Mingjian,

    Could you try building the kernel driver into the kernel instead of as a kernel module for both EP and RC? And do a power-on sequence of booting up EP first and configuring it fully, then after EP is up, turn on the RC?

    Above should make it so that the RC should detect the EP during boot instead of having to manually run a rescan. The hotplug feature for the EP/RC is not robust.

    Regards,

    Takuma

  • Hi  Takuma, 

    I tried to build drivers into kernel and reboot RC after EP configured. Seems that there is no difference. 

    Regards,

    Mingjian

  • Hi Mingjian,

    Thanks for trying. As for the memory barrier and cache coherency suspicion, were you able to try this out?

    Regards,

    Takuma

  • Hi  Takuma, 

    Is there any update on this issue?

    By the way, I tried to use memory barrier and  it did not work. 

    Regards,

    Mingjian

  • Hi Mingjian,

    If manually invalidating cache does not work, I propose you handle within application.

    Regards,

    Takuma