• State TI Thinks Resolved
  • Locked Locked
  • Replies 7 replies
  • Answers 2 answers
  • Subscribers 99 subscribers
  • Views 652 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

Linux/PROCESSOR-SDK-AM65X: AM6 - Linux SDK 5.2 Giga Ethernet testing fail on PRU (Eth1 / Eth2)

Linux/PROCESSOR-SDK-AM65X: SDK 5.3 problems with Ethernet

Rio Chan
Genius 14935 points
Part Number: PROCESSOR-SDK-AM65X

Tool/software: Linux

Hi.

We have tested the SDK5.3, the Ethernet is still buggy.

please see the below log.

Thanks.

BR Rio

root@am65xx-evm:~# iperf -c 10.1.42.11 -w128K -d -i1 -t5

------------------------------------------------------------

Server listening on TCP port 5001

TCP window size:  256 KByte (WARNING: requested  128 KByte)

------------------------------------------------------------

------------------------------------------------------------

Client connecting to 10.1.42.11, TCP port 5001

TCP window size:  256 KByte (WARNING: requested  128 KByte)

------------------------------------------------------------

[  5] local 10.1.42.24 port 60964 connected with 10.1.42.11 port 5001

[  4] local 10.1.42.24 port 5001 connected with 10.1.42.11 port 40121

[ ID] Interval       Transfer     Bandwidth

[  5]  0.0- 1.0 sec  3.12 MBytes  26.2 Mbits/sec

[  4]  0.0- 1.0 sec  92.1 MBytes   773 Mbits/sec

[  258.844909] skbuff: skb_over_panic: text:ffff0000014e2b54 len:1584 put:1584 head:ffff8008440b5000 data:ffff8008440b5082 tail:0x6b2 end:0x680 dev:eth4

[  258.858382] ------------[ cut here ]------------

[  258.862992] kernel BUG at /oe/bld/build-AARCH64_1/arago-tmp-external-linaro-toolchain/work-shared/am65xx-evm/kernel-source/net/core/skbuff.c:104!

[  258.876009] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP

[  258.881482] Modules linked in: sha512_arm64 md5 des_generic cbc xhci_plat_hcd xhci_hcd usbcore rpmsg_proto virtio_rpmsg_bus rpmsg_core xfrm_user dwc3 udc_core usb_common ti_am335x_adc kfifo_buf xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key xfrm_algo ti_k3_r5_remoteproc pruss_soc_bus pvrsrvkm(O) pci_endpoint_test m_can dwc3_of_simple omap_rng phy_omap_usb2 rng_core sa2ul can_dev ti_am335x_tscadc at24 authenc icssg_prueth pru_rproc pruss pruss_intc remoteproc crc32_ce gpio_decoder crct10dif_ce input_polldev sch_fq_codel cryptodev(O) ipv6

[  258.929835] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G           O    4.14.79-ge669d52447 #1

[  258.938079] Hardware name: Texas Instruments AM654 Base Board (DT)

[  258.944246] task: ffff000008b3b280 task.stack: ffff000008b00000

[  258.950162] PC is at skb_panic+0x48/0x50

[  258.954075] LR is at skb_panic+0x48/0x50

[  258.957987] pc : [<ffff000008683c38>] lr : [<ffff000008683c38>] pstate: 20000145

[  258.965363] sp : ffff00000800fd80

[  258.968666] x29: ffff00000800fd90 x28: ffff0000087e0110

[  258.973968] x27: ffff800841ffd9c0 x26: ffff8008443ee500

[  258.979269] x25: ffff800845c5c200 x24: ffff800841ffdaf0

[  258.984571] x23: 0000000000000040 x22: 0000000000000630

[  258.989872] x21: ffff800841ffd900 x20: ffff800841ffd000

[  258.995173] x19: 0000000000000000 x18: 0000000000000010

[  259.000474] x17: 0000ffff9f140810 x16: ffff00000867cf20

[  259.005776] x15: ffffffffffffffff x14: 30383678303a646e

[  259.011077] x13: 652032623678303a x12: ffff000008b33a30

[  259.016378] x11: ffff000008478a98 x10: 38666666663a6174

[  259.021680] x9 : 0000000000000016 x8 : 3034343830303866

[  259.026981] x7 : 6666663a64616568 x6 : 00000000000001c2

[  259.032284] x5 : 0000000000000001 x4 : 0000000000000004

[  259.037584] x3 : 0000000000000000 x2 : 0000000000000040

[  259.042885] x1 : ffff000008b3b280 x0 : 0000000000000089

[  259.048189] Process swapper/0 (pid: 0, stack limit = 0xffff000008b00000)

[  259.054873] Call trace:

[  259.057314] Exception stack(0xffff00000800fc40 to 0xffff00000800fd80)

[  259.063743] fc40: 0000000000000089 ffff000008b3b280 0000000000000040 0000000000000000

[  259.071557] fc60: 0000000000000004 0000000000000001 00000000000001c2 6666663a64616568

[  259.079370] fc80: 3034343830303866 0000000000000016 38666666663a6174 ffff000008478a98

[  259.087184] fca0: ffff000008b33a30 652032623678303a 30383678303a646e ffffffffffffffff

[  259.094997] fcc0: ffff00000867cf20 0000ffff9f140810 0000000000000010 0000000000000000

[  259.102810] fce0: ffff800841ffd000 ffff800841ffd900 0000000000000630 0000000000000040

[  259.110625] fd00: ffff800841ffdaf0 ffff800845c5c200 ffff8008443ee500 ffff800841ffd9c0

[  259.118439] fd20: ffff0000087e0110 ffff00000800fd90 ffff000008683c38 ffff00000800fd80

[  259.126253] fd40: ffff000008683c38 0000000020000145 ffff0000014e2b54 0000000000000630

[  259.134067] fd60: 0000ffffffffffff ffff8008440b5000 ffff00000800fd90 ffff000008683c38

[  259.141881] [<ffff000008683c38>] skb_panic+0x48/0x50

[  259.146835] [<ffff000008685b20>] pskb_put+0x0/0x38

[  259.151647] [<ffff0000014e2b54>] emac_napi_rx_poll+0x1b4/0x2b0 [icssg_prueth]

[  259.158772] [<ffff00000869df54>] net_rx_action+0xf4/0x2b0

[  259.164162] [<ffff0000080811ec>] __do_softirq+0x12c/0x228

[  259.169551] [<ffff0000080acd10>] irq_exit+0xc8/0xf8

[  259.174421] [<ffff0000080fd8c0>] __handle_domain_irq+0x60/0xb0

[  259.180242] [<ffff000008080fc4>] gic_handle_irq+0x7c/0x178

[  259.185715] Exception stack(0xffff000008b0fdf0 to 0xffff000008b0ff30)

[  259.192141] fde0:                                   0000000000000000 00008008773c0000

[  259.199954] fe00: 0000000000000000 ffff000008af3930 ffff000008af3908 ffff000008b0ff20

[  259.207769] fe20: 00008008773c0000 0000000000000000 0000000000000002 ffff000008b0feb0

[  259.215582] fe40: 0000000000000920 0000000000000001 000000396684b913 0000000000000000

[  259.223396] fe60: 002bcca204681cc1 0000560bf0616fcb ffff00000867cf20 0000ffff9f140810

[  259.231210] fe80: 000000000000010f ffff000008b33ac4 ffff000008b33a30 ffff000008af0018

[  259.239023] fea0: ffff000008b3b280 ffff80087ffff480 ffff000008ae0028 0000000000000000

[  259.246837] fec0: 00000000ffef3ce0 0000000000000400 0000000080a90018 ffff000008b0ff30

[  259.254650] fee0: ffff000008084e2c ffff000008b0ff30 ffff000008084e30 0000000060000145

[  259.262465] ff00: 0000000000000000 00000000ffef3ce0 ffffffffffffffff ffff000008123f84

[  259.270278] ff20: ffff000008b0ff30 ffff000008084e30

[  259.275145] [<ffff000008082a30>] el1_irq+0xb0/0x124

[  259.280013] [<ffff000008084e30>] arch_cpu_idle+0x10/0x18

[  259.285313] [<ffff0000080e5ee0>] do_idle+0xd8/0x118

[  259.290180] [<ffff0000080e60b4>] cpu_startup_entry+0x24/0x28

[  259.295831] [<ffff00000879621c>] rest_init+0xcc/0xd8

[  259.300792] [<ffff000008a90b50>] start_kernel+0x378/0x38c

[  259.306183] Code: a90023eb aa0a03e1 912f6120 97e9e357 (d4210000)

[  259.312271] ---[ end trace 3026f6f051d27fb8 ]---

[  259.316877] Kernel panic - not syncing: Fatal exception in interrupt

[  259.323216] SMP: stopping secondary CPUs

[  259.327139] Kernel Offset: disabled

[  259.330618] CPU features: 0x080200c

[  259.334096] Memory Limit: none

[  259.337144] ---[ end Kernel panic - not syncing: Fatal exception in interrupt

  • 0
    TI__Guru** 102120 points
    Hello Rio,

    Thanks for letting us know. I will try to replicate your results on this side and report it. Could you tell me more about your hardware setup?

    Regards,
    Nick
  • 0 in reply to Nick Saulnier
    Expert 2620 points
    Hi Nick

    AM65xIDK [eth6] <---> giga switch <---> Host PC
  • 0 in reply to WTMEC_Will Hsu
    TI__Genius 14935 points

    Hi.

    We have tested the SDK 5.3.

    We found the Ethernet issue is still there.

    Here are the results:

    Testing on the GP EVM:

           A. Testing the GP EVM, all Ethnet can works, but once the Cable re-plug in from A port to B port, the A port Assign IP will be gone.

           B. Testing the GP EVM, the Throughput on the Eth1/Eth2 are not coming up ~1GMbps.

           C. Testing with Iperf2 will get the Netowkr error, not reachable.

    Testing the IDK EVM:

           A. Testing the IDK EVM, only the Eth0 poart can be tested.

           B. Once re-plug the Ethernet Cable from A port to B port, the Assigned IP will be gone.

           C. All the Eth1~6 are not stable, and even we re-assign the IP for each of them, once the Cable is re-plug in from A to B, the Ping/ Connetion / Iperf are not working.

           D. Only the Eth0 port are stable.

       This kind of Quality is not acceptable, please fix.

        Thanks.

    BR Rio

  • 0 in reply to Rio Chan
    TI__Guru** 102120 points

    Hello Rio,

    I am sorry for the delayed response. I am able to replicate your kernel panic for bidirectional communication in SDK 5.3. That kernel panic should be fixed in the next SDK release, which is coming out in the June/July timeframe. The team is currently working through another bug associated with bidirectional communication. At this point in time, I cannot comment on if all known bugs will be resolved in the next release or not.

    Regards,

    Nick

  • 0 in reply to Nick Saulnier
    TI__Guru** 102120 points
    Hello Rio,

    I am going to split this into two posts so that it is easier to read. Post 1 will reply to your previous post. Post 2 will allow you to make sure I understand all of the issues you are observing.

    POST 1

    I do not have any more updates at this time on the bidirectional communication bugs.

    I have seen the iperf "not reachable" error when trying to connect to a port that no longer has an IP address assigned to it. I wonder if that is the cause of that error you are observing.

    On the IP address issue: How are you assigning IP addresses?

    * If I statically assign an IP address during boot up, then I expect the port to keep that IP address regardless of how many times I plug and unplug ethernet cables into the port.

    * If I assign an IP address using a method like "ifconfig eth2 192.168.2.100 netmask 255.255.255.0 up", then I would expect that IP address to go away when I unplug a cable from that port. That way, the port can have a new IP address assigned to it if it is connected to a DHCP server.

    * If you are getting the IP address assigned by a DHCP server, I would still expect the IP address to go away when you unplug the cable. When you reconnect to the DHCP server, you might get reassigned the same IP address - however, you are not guaranteed to get the same IP address.

    Regards,
    Nick
  • 0 in reply to Nick Saulnier
    TI__Guru** 102120 points
    POST 2

    Let me know if this is an accurate summary of the issues you are observing in SDK 5.3. Please correct me or add anything that needs to be added:

    1) Kernel panics are observed in bidirectional iperf testing. (The kernel panics should be fixed in next SDK, but additional bugs associated with bidirectional testing may or may not be addressed)

    2) Throughput is low. (the throughput should improve in future releases, but at this point in time I cannot promise that all throughput issues will be addressed in the next SDK)

    3) There may or may not be something going on with IP addresses. Please see post 1 for more information.

    Regards,
    Nick
  • 0 in reply to Nick Saulnier
    TI__Guru** 102120 points
    Hello Rio,

    Thank you for bringing up these Ethernet bugs. I am going to close this ticket on my side. Feel free to reply if we need to continue the conversation.

    Regards,
    Nick