• State TI Thinks Resolved
  • Locked Locked
  • Replies 14 replies
  • Answers 2 answers
  • Subscribers 99 subscribers
  • Views 1315 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM5726: PRU Ethernet firmware repository

Michael Shklyarman
Genius 12850 points

Part Number: AM5726

Tool/software: Linux

Champs,

customer is using the latest 4.9.82-rt76 kernel and experience some issues in PRU ethernet functionality. In the 4.9 based PSDK the exact kernel version is 4.9.65. the questions I have are:

1. are there any changes in PRU Ethernet driver/firmware between 4.9.65 and 4.9.82

2. where PRU Ethernet firmware is maintained?

3. What is the right way to get the PRU Ethernet firmware associate with 4.9.82 rt kernel

thanks

Michael

  • 0
    TI__Guru** 108980 points
    Hello Michael,

    What issues is the customer experiencing? Can they replicate it using one of TI's software releases? PRU Ethernet is still under active development, so their issues might have been fixed by a later version of code.

    Background:
    It looks like we released SDKs with Linux 4.9.59 (SDK 4.02), Linux 4.9.69 (SDK 4.03), and Linux 4.14.40 (SDK 5.0). However, TI has not released a version of Linux 4.9.65. We provide a new PRU Ethernet binary file to use with each SDK release. To the best of my knowledge, we do not expose the PRU Ethernet firmware source code to customers for those SDKs.

    Regards,
    Nick
  • 0 in reply to Nick Saulnier
    TI__Genius 12850 points
    Hi Nick,

    I looked at the top of the Linux Makefile in the PSDK 4.3.0.5 and it says VERSION=4/PATCHLEVEL= 9/SUBLEVEL =65. the name of the folder itself actually says 4.9.69. Not sure what is the reason for the discrepancy, but it probably doesn't matter.
    The point is, that the customer is using 4.9.82 that they took directly off the TI repository. They do not observe any issues running stock SDK on IDK board. My suspicion is that the PRU Ethernet kernel driver has evolved between 4.9.69 and 4.9.82 and the firmware released with SDK 4.3.0.5 is not fully compatible with the driver code. The question I have is whether the driver had indeed evolved and if yes, where to get compatible PRU firmware. below is the description of the customer's setup and the issue they observe for reference

    • We use our own application to send broadcast packet on the PRU ethernet port and loop the packet back in using a loopback stub on the ethernet port.
    • We are sending 1500 bytes packets and a burst of 666 packets at about 5.04 Mbps.
    • The PRU ethernet port is setup for 100Mbps.
    • Sending these bursts continuously, we see sometimes that we don’t receive the 666 packets back. We would see a loss of about 1-20 packets sometimes.
    • On a test run of about 1000 runs, we observe packet losses on about 35 of those.
    • Using ethtool, I see that there is a difference between the Txbroadcast and Rxbroadcast counts, where Txbroadcast count is higher than Rxbroadcast count. It seems to suggest that the driver didn’t receive the packets.
    • Running the same test on the GMAC path, we observe no packet drops.
    • We have tried changing the stubs and our boards, but we observe the same issue on all the boards.

    thanks
    Michael
  • 0 in reply to Michael Shklyarman
    TI__Genius 12850 points

    Hi Nick,

    Any guidance for us here?

    thanks

    Michael

  • 0 in reply to Michael Shklyarman
    TI__Guru** 108980 points
    Hello Michael,

    Which TI repo are they using? Can we get the commit ID they are using from the repo? The repo could be the source of their problem.

    For that version of Linux, I would expect that the customer would want to use the firmware from Processor SDK 4.3.

    Regards,
    Nick
  • 0 in reply to Nick Saulnier
    TI__Genius 12850 points

    Nick,

    Could you please provide link to the TI git repo with 4.9.x rt kernel? the story on customer side is rather complicated, i would like to get back to basics first

    thanks

    Michael

  • 0 in reply to Michael Shklyarman
    TI__Guru** 108980 points

    Hello Michael,

    There are two primary TI git repos: ti-linux-kernel, and processor-sdk-linux. ti-linux-kernel is for TI's development team. processor-sdk-linux is used in our Processor SDKs, and is for TI customers. You can clone processor-sdk-linux from git://git.ti.com/processor-sdk/processor-sdk-linux.git .

    As a general rule, I do not support ti-linux-kernel. That is because processor-sdk-linux adds a bunch of patches on top of ti-linux-kernel. The prueth driver in particular has significant differences between ti-linux-kernel and processor-sdk-linux, so I would expect it to behave differently between the two repos. I would point your customers to branch processor-sdk-linux-rt-04.03.00 if they want to be supported on this forum.

    Based on your post, I assume your customer is using ti-linux-kernel, branch ti-rt-linux-4.9.y . Keep in mind that the tag ti2017.06 was used to create RT Linux SDK 4.3. TI switched our active development to Linux 4.14 at that point. That means any commits in branch ti-rt-linux-4.9.y after the ti2017.06 tag are just pulled in from mainline. Those later commits are not tested or validated by TI.

    Another note: sorry for the confusion on Linux number. I forgot Linux SDK 4.3 used 4.9.69, while RT Linux SDK 4.3 used 4.9.65.

    Regards,
    Nick

  • 0 in reply to Nick Saulnier
    TI__Genius 12850 points

    Nick,

    the processors sdk RT 4.3.0 is using rt patch version -rt23. this seems to be rather old. Do you know if there is any particular reason for using this version of patch set with 4.9.65 kernel?

    thanks

    Michael

  • 0 in reply to Michael Shklyarman
    TI__Genius 12850 points

    Nick,

    The customer tested with SDK kernel on their HW and reported good performance with original issue gone. Thank you for your clarifications.

    They did observe number of out-of-order packets received in the loopback test. Do you know what could cause this? Are there multiple Rx queues? I'll appreciate your insights

    thanks

    Michael

  • 0 in reply to Michael Shklyarman
    TI__Guru** 108980 points
    Hello Michael,

    To verify: These are the PRUETH ports and not the CPSW ports? Which loopback test is being run? Is this on TI EVMs, if so which EVMs?

    Regards,
    Nick
  • 0 in reply to Nick Saulnier
    TI__Genius 12850 points
    Nick,

    The test runs on PRU port on customer's HW using their own test program. Loopback is achieved using RJ-45 stub plugged into the PRU port that loops back outgoing traffic.

    thanks
    Michael
  • 0 in reply to Michael Shklyarman
    TI__Guru** 108980 points
    Hello Michael,

    On rt patch version: rt23 is the latest patch TI will support for Linux 4.9 / RT Linux SDK 4.3. If the customer is interested in later RT patches, we suggest upgrading to a later SDK.

    I am still working on the question about out-of-order packets.

    Regards,
    Nick
  • 0 in reply to Nick Saulnier
    TI__Guru** 108980 points

    Hello Michael,

    We do have multiple RX queues. Please see this section of our documentation for the PRU Ethernet driver. . This section on performance test and logs includes one method to avoid out of order packets by steering frames to a single CPU.

    Future readers, please note that the PRU Ethernet documentation will be re-organized sometime in the future. The links above may or may not work going forward. In general, PRU Ethernet information will be located somewhere under PRU-ICSS / PRU_ICSSG or   Industrial Protocols. 

    Regards,

    Nick

  • 0 in reply to Nick Saulnier
    TI__Guru** 108980 points
    Hello Michael,

    I am going to mark this thread resolved. Feel free to reply to continue the discussion, or create a new thread if a different subject comes up.

    Regards,
    Nick
  • 0 in reply to Nick Saulnier
    TI__Guru** 108980 points
    Additional information on Receive Packet Steering in the Linux kernel source under Documentation/networking/scaling.txt