AM6442: prueth with 10 MBit/s RGMII: transmit queue 0 timed out

Steffen,

Is this with ICSSG based Ethernet using the board detailed in https://www.ti.com/tool/TIDA-010262 ? 10M should be supported, but testing might be limited as we don't have DP83TD510E 10 BASE-T1L Ethernet PHY board in the test environment.

Steffen Graf70 said:

The Ethernet connections is working fine as long as the link remains stable, after the one or the other link drop, it does not resume well. 

So the one orther link drop refers to the two ports from the same ICSSG subsystem?

  Pekka

Hi Pekka,

yes, it happens on TIDA-010262, as well as TIDA-010261 where we only have one port. I don't see any correlation between the ports, more it is just random after loosing link, if it will be working when the link comes back again.

Best regards,

Steffen

Can you clarify what you think is a bug here? I'll type my understanding from above. This is with ICSSG based Ethernet and DP83TD510E 10 BASE-T1L Ethernet PHY. The driver for DP83TD510E, is this an upstream driver or something experimental?

1. You stop sending any frames (TX_en line of the RGMII interface is not going high any more and no frames are sent out)

2. Some amount of time expires. Any detail on how long?

3. Kernel crash with "NETDEV WATCHDOG: eth0 (icssg-prueth): transmit queue 0 timed out"

There also is some mention of dependency with the other port of the ICSSG, but lets start with the single port case.

Hi Pekka,

the driver for DP83TD510E is from upstream 6.1 with some modifications in the init sequence.

The problem is the "NETDEV WATCHDOG: eth0 (icssg-prueth): transmit queue 0 timed out". As long as the Ethernet link is stable, this error does not occur. But when loosing the link and reestablishing it, it comes to this error. 

Sometimes it does take not only one, but maybe two or three link losses and restores until it occurs.

Best regards,

Steffen

Steffen Graf70 said:

But when loosing the link and reestablishing it

What does this mean? Link goes down and back up? What is the relationship to the mention earlier of no transmit packets?

Exactly, link goes down, and back up. For example when unplugging a cable or having an ESD/EFT event. 

After this Linux still continues to send packets (shown in tcpdump), but these packets are not seen on the RGMII interface. 

Best regards,

Steffen

To narrow down can we try see if this is tied to the  DP83TD510E driver or just ICSSG Ethernet in general. Can you try to repeat link up/down and ICSSG Ethernet on EVM or SK to see if you see the same error?

Also what are you using to send frames? Usually the application socket will stop on link down, and there will be a sequence to re-open the socket and start sending.

I will test this with the EVM and try to force one port to 10 Mbit/s. 

for testing I'm using just a simple ping.

At first even 1G on ICSSG on EVM would be interesting. Just to narrow down,

ping will exit with Network unreachable on link down? So do you restart it? Can you attach the commands and printouts you see on the console?

Ping will not exit (on the machine it is running, it shows "Destination Host Unreachable", on the other one, there is no output, as answering to a ping is done by the Kernel) as far as I remember, I can check this in the next days on the board and share the console output.

I just tried:

1. start ping on eth2 (ICSSG) to a Linux desktop in the local LAN

2. unplug the Ethernet cable

3. ping exits Network unreachable

So are you running ping on the remote side, the AM64x ICSSG Eth port is the one replying?

Hi Pekka, you are using the Yocto built image, right? Debian ping seems to behave different. 

Steffen Graf70 said:

you are using the Yocto built image, right? Debian ping seems to behave different.

So your setup is not what I described but instead:

1. Start ping to AM64x ICCSG on a Debian desktop (ping is running on the remote device, not the local AM64x where you see issues)

2. unplug cable, or introduce errors

3. Local ICSSG eth that is responding to pings, stops transmitting, looks like Linux still thinks it is sending over ICSSG eth but frames never show up at xMII interface

Hi Pekka,

I am able to reproduce this behavior on the EVM using tisdk-debian-bookworm-am64xx-evm.wic.xz and added the latest ti-linux-firmware to have ti-pruss firmware in there.

1. connect eth2 to a network

2. get an IP (dhclient -v eth2)

3. install ethtool (apt install ethtool)

4. force eth2 to 10 Mbit/s mode (ethtool -s speed 10)

5. ping ti.com (you can speed this up using ping -f, but please ping a local machine and not ti.com ;-) ) or ping the EVM from another machine

6. unplug ethernet cable

7. if ping resumes normally, goto 6

8. wait until it crashes

With the Arago image tisdk-default-image-am64xx-evm.wic.xz, it shows the same behavior, but ping exits when unplugging the cable. 

Best regards,

Steffen

Hi Stefan,

I have connected the board and PC to same network and followed the steps you mentioned.

I am working with this image: tisdk-default-image-am64xx-evm.wic.xz

Observations:

• PC (Transmitter) -> AM64 board (Receiver) -> NO crash seen, PC halts for the moment and again starts to transmit on plugging back in.
• AM64 (Transmitter) -> PC (Receiver) -> As soon as link goes down that is disconnected, Throws Network is Unreachable

 I'll discuss with dev team and will come back to you.

Thanks and Regards,

Rimika

Did you try unplug and plug several times? It sometimes takes 10 or more times. Maybe with ping -f or iperf it goes faster.

Hi Stefan,

I have connected the board and PC to same network and followed the steps you mentioned.

I am working with this image: tisdk-default-image-am64xx-evm.wic.xz

Observations:

• PC (Transmitter) -> AM64 board (Receiver) -> NO crash seen, PC halts for the moment and again starts to transmit on plugging back in.
• AM64 (Transmitter) -> PC (Receiver) -> As soon as link goes down that is disconnected, Throws Network is Unreachable

 I'll discuss with dev team and will come back to you.

Thanks and Regards,

Rimika

Hi,

do you have any update?

Best regards,

Steffen