This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TMDSIDK574: PRU Ethernet ping issue

Part Number: TMDSIDK574

This is the extension of this thread:
e2e.ti.com/.../3105350

Hardware:
57xx IDK, pru-ethernet via prp (eth2 and eth3).
the san-device (IBM PC).
The IDK pings the san.

I have install the packages for initialize the pru:
1. Copy that libs into the rootfs/usr/lib:
libarchive.so
libarchive.so.13
libarchive.so.13.3.2
libbz2.so
libbz2.so.1
libbz2.so.1.0.6
liblzo2.so
liblzo2.so.2
liblzo2.so.2.0.0
libsolv.so
libsolv.so.0

2. Copy the opkg into the rootfs.

3. Copy that ipk into the rootfs/home/root:
kernel-module-pru-rproc-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
kernel-module-pruss-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
kernel-module-pruss-intc-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
kernel-module-pruss-soc-bus-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
kernel-module-remoteproc-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
kernel-module-ti-prueth-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
prueth-fw_5.1.4-r0_am57xx_evm.ipk
iproute2_4.11.0-r0.arago3_armv7ahf-neon.ipk
libelf1_0.170-r0_armv7ahf-neon.ipk
and install them at linux arago:
opkg --add-arch am57xx_evm:66 install --force-depends kernel-module-pru-rproc-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
opkg --add-arch am57xx_evm:66 install --force-depends kernel-module-pruss-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
opkg --add-arch am57xx_evm:66 install --force-depends kernel-module-pruss-intc-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
opkg --add-arch am57xx_evm:66 install --force-depends kernel-module-pruss-soc-bus-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
opkg --add-arch am57xx_evm:66 install --force-depends kernel-module-remoteproc-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
opkg --add-arch am57xx_evm:66 install --force-depends kernel-module-ti-prueth-4.14.67-gd315a9bb00_4.14.67+git0+d315a9bb00-r7a.arago5.tisdk1.3_am57xx_evm.ipk
opkg --add-arch am57xx_evm:66 install --force-depends prueth-fw_5.1.4-r0_am57xx_evm.ipk
opkg --add-arch armv7ahf-neon:56 install --force-depends libelf1_0.170-r0_armv7ahf-neon.ipk
opkg --add-arch armv7ahf-neon:56 install --force-depends iproute2_4.11.0-r0.arago3_armv7ahf-neon.ipk

I have no make more any change in sdk.

So, after the 2nd compilation of sdk, there are are the same bug in the ping: the ping is unstable.

Firstly, the ping is great, up to 10ms.
Then ping gradually decreases downto 0.7ms, and then is again 10ms.
And again.

When IDK pings the 127.0.0.1, the ping is 0.06ms stable.

Please help.

  • Hello,

    1) What version of Processor SDK are you using?

    2) What results do you see if you use an out-of-the-box SD card? (run bin/create-sdcard.sh)

    Regards,

    Nick

  • Nick Saulnier, hi!

    1. I have use the commands:

    wget --no-check-certificate releases.linaro.org/.../gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf.tar.xz
    wget --no-check-certificate releases.linaro.org/.../gcc-linaro-7.2.1-2017.11-x86_64_aarch64-linux-gnu.tar.xz

    tar -Jxvf gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf.tar.xz -C $HOME
    tar -Jxvf gcc-linaro-7.2.1-2017.11-x86_64_aarch64-linux-gnu.tar.xz -C $HOME
    git clone git://arago-project.org/git/projects/oe-layersetup.git tisdk

    cd tisdk
    ./oe-layertool-setup.sh -f configs/processor-sdk/processor-sdk-05.01.00.11-config.txt

    2. I originally used an micro-sd card (sorry I didn't tell you).

  • Hello,

    We want to cut out all of your steps above to make sure they aren't introducing a problem somewhere.

    Please download the default SDK. You can find the AM57xx Linux Processor SDK 5.1 here.

    In your SDK, run bin/create-sdcard.sh to create your SD card. Re-run the ping tests to see if you have the same behavior.

    Regards,

    Nick

  • Nick Saulnier, hi.

    After installing, your sdk (from your link):
    ti-processor-sdk-linux-am57xx-evm-05.01.00.11
    my sdk:
    ti-processor-sdk-linux-rt-am57xx-evm-05.01.00.11

    I have made the sd-card by bin/create-sdcard.sh -> check the default boot and rootfs.
    Then, save the default-boot partition and re-run the bin/create-sdcard.sh -> check the manual default-boot and my compiled rootfs.
    The result is the same:

    Please help.

  • Hello,

    I am not able to replicate your results. I ran my tests using NFS instead of an SD card, but the SD card should actually perform better in networking speed tests.

    Are you using Linux, or RT Linux?

    Here are my results: (the same results were observed with Linux SDK 5.1 and Linux SDK 6.0. I did not test RT Linux)

    // This is from the AM572x IDK terminal
    am57xx-evm login: root
    root@am57xx-evm:~# uname -a
    Linux am57xx-evm 4.14.67-gd315a9bb00 #1 SMP PREEMPT Sat Oct 6 02:39:21 UTC 2018 armv7l GNU/Linux
    root@am57xx-evm:~# ifconfig eth2 192.168.1.151
    [   46.970546] remoteproc remoteproc6: powering up 4b2b4000.pru
    [   46.978427] remoteproc remoteproc6: Booting fw image ti-pruss/am57xx-pru0-prueth-fw.elf, size 5028
    [   46.987768] pruss 4b280000.pruss: configured system_events[63-0] = 0x00000600.00500000
    [   46.995863] pruss 4b280000.pruss: configured intr_channels = 0x00000095 host_intr = 0x00000115
    [   47.004631] remoteproc remoteproc6: remote processor 4b2b4000.pru is now up
    [   47.011697] net eth2: started
    [   47.018919] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready
    [   47.024844] prueth pruss2_eth eth2: Link is Up - 100Mbps/Full - flow control rx/tx
    [   47.032891] IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
    
    
    // this is from the Linux PC
    $ ping 192.168.1.151
    PING 192.168.1.151 (192.168.1.151) 56(84) bytes of data.
    64 bytes from 192.168.1.151: icmp_seq=1 ttl=64 time=0.115 ms
    64 bytes from 192.168.1.151: icmp_seq=2 ttl=64 time=0.148 ms
    64 bytes from 192.168.1.151: icmp_seq=3 ttl=64 time=0.155 ms
    64 bytes from 192.168.1.151: icmp_seq=4 ttl=64 time=0.151 ms
    64 bytes from 192.168.1.151: icmp_seq=5 ttl=64 time=0.147 ms
    64 bytes from 192.168.1.151: icmp_seq=6 ttl=64 time=0.146 ms
    64 bytes from 192.168.1.151: icmp_seq=7 ttl=64 time=0.147 ms
    ^C
    --- 192.168.1.151 ping statistics ---
    7 packets transmitted, 7 received, 0% packet loss, time 6139ms
    rtt min/avg/max/mdev = 0.115/0.144/0.155/0.013 ms

    My process:

    I created an SD card using the create-sdcard.sh script in the SDK. Note that I used the "tisdk-rootfs-image" filesystem, not the docker filesystem. I then used setup.sh in the base directory of the SDK to install the same filesystem on my computer. The Linux PC, the AM572x IDK CPSW port (used for NFS), and the AM572x IDK PRUETH port were all connected to the same gigabit switch.

    Regards,

    Nick

  • Nick Saulnier, hi.

    Note that my bad ping is from IDK to the PC.

    Note that I have compile the sdk according to the "load time minimization", according to the thread e2e.ti.com/.../3105350

    If I understand you correctly, you want me to step away from this.
    But in this case I will lose bonuses from thread e2e.ti.com/.../3105350

    Perhaps I should have said that it is imperative to use my rootfs to ensure minimum boot time.
    I was hoping to fix the ping without changing the rootfs, for example by installing any packages.
    Could you tell me please, in what direction I should go.

    Please help.

  • Hello Leshka,

    If I can replicate your observations, then it is easier for me to help you. All the extra steps from your other threads may not impact on ping performance. But first, let's make sure we are seeing the same thing with the simplest use case. Then we can start adding complexity.

    1) Are you using RT Linux or regular Linux? I want to make sure we are using the same SDK.

    2) Using the out-of-the-box filesystem (from create-sdcard.sh), please show me your ping results a) from the PC when you ping the IDK, and b) from the IDK when you ping the PC.

    Regards,

    Nick

  • Nick Saulnier, hi.

    1. My using sdk is ti-processor-sdk-linux-rt-am57xx-evm-05.01.00.11. Note that еhe method I described in the previous comment, I used with RT Linux create-sdcard.sh (I have made the sd-card by bin/create-sdcard.sh -> check the default boot and rootfs. Then, save the default-boot partition and re-run the bin/create-sdcard.sh -> check the manual default-boot and my compiled rootfs).

    2. If I use the out-of-the-box filesystem (from create-sdcard.sh) Linux or "RT Linux", the ping is stable and quick.

    Please help.

  • Nick Saulnier, hi.

    Perhaps this my information will help you.

    1. I prepare the sd-card using the out-of-the-box "RT Linux" (create-sdcard.sh).
    2. At sd-card, I have replace there rootfs/boot on rootfs/boot from my compiled sdk.
    Result: the ping has become just as bad.
    As I understood, the bug is in rootfs/boot/image.

    Please help.

  • Hello,

    1) I want to make sure we understand which tests you have run. Is this correct? If not, please correct me:

    1a) You used create-sdcard.sh to install out-of-the-box RT Linux SDK 5.1 in the SD card. At this point, PRU Ethernet ping works as expected. You copied the rootfs/boot directory generated by your compiled SDK. You deleted the rootfs/boot directory from the SD Card. You pasted the compiled rootfs/boot directory onto the SD card. Now, PRU Ethernet ping behaves like the screenshot you posted on September 26.

    1b) Have you tested ping with CPSW? What is the CPSW performance?

    Regards,

    Nick

  • Nick Saulnier, hi.

    1a) Yes, it is. Note that I have setup the PRU by installing the packages (posted on the top of thread).
    1b)
    - The out-of-the-box RT Linux SDK 5.1 in the SD card, CPSW-prp: the ping is stable (2.3ms).
    - The out-of-the-box RT Linux SDK 5.1 in the SD card, CPSW-prp, installed pru-packages: the ping is stable and quick (1.5ms).

    Please help.

  • Hello,

    Our current guess is that something got messed up in your kernel config. When PRU Ethernet is pinging something else, the scheduler in regular Linux is expected to yield PRU Ethernet ping times anywhere between 0-10ms. However, the RT Linux scheduler should give consistently low ping times.

    1) Could you test this case? Build the out-of-the-box RT Linux kernel and install the kernel and kernel modules in your tiny filesystem. You can find steps to build and install the kernel and modules in our Kernel User's Guide. (Future readers: I linked to the SDK 5.1 version of documentation. You can find the Kernel User's Guide for the current version of documentation here.)

    It might be useful to take a diff between the out-of-the-box .config file and the .config file from your original kernel.

    Regards,

    Nick

  • Hello,

    I am going to mark this thread resolved for now. Feel free to reply if you would like to continue the conversation.

    Regards,

    Nick

  • Nick Saulnier, hi.

    Sorry for the delay.

    I will answer you soon.

  • Nick Saulnier, one small clarification for me:
    As I understand it, I have to install those packages that I installed in rt linux of my compiled sdk?

  • Hello,

    If you are talking about the use case "Build the out-of-the-box RT Linux kernel and install the kernel and kernel modules in your tiny filesystem", then you do not need to also install all the packages you listed in your first post on this thread. The PRU drivers and associated code should be there.

    Regards,

    Nick

  • Nick Saulnier, hi!

    I have compile the image:

    export PATH=/home/vl/ti-processor-sdk-linux-rt-am57xx-evm-05.01.00.11/linux-devkit/sysroots/x86_64-arago-linux/usr/bin:$PATH
    cd /home/vl/ti-processor-sdk-linux-rt-am57xx-evm-05.01.00.11/board-support/linux-rt-4.14.67+gitAUTOINC+cfb76a2032-gcfb76a2032
    
    make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- tisdk_am57xx-evm-rt_defconfig
    make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage
    
    rm /media/vl/rootfs/boot/zImage-4.14.67-rt40-gcfb76a2032
    cp /home/vl/ti-processor-sdk-linux-rt-am57xx-evm-05.01.00.11/board-support/linux-rt-4.14.67+gitAUTOINC+cfb76a2032-gcfb76a2032/arch/arm/boot/zImage /media/vl/rootfs/boot/zImage-4.14.67-rt40-gcfb76a2032
    sync
    

    Then, I have setup the pru-prp.

    And the ping is stable and quick.

    P.S. I first incorrectly assumed that you want to check if the cause of the bug is installed ipk.

    Please help

  • Hello,

    Ok, we now know that the issue is with how you configured your kernel, not with the tiny filesystem itself. Is the default kernel + tiny filesystem small enough for your usecase? If not, I would start cutting out kernel modules that you do not need until you get a working RT Linux kernel that is the right size for your system.

    Regards,

    Nick

  • Fore future readers: post split, discussion about improving boot times can be seen in the follow-up thread.

    Regards,

    Nick