AM3354: Eth1 not working

Paolo Bartolini

Part Number: AM3354

Hi,
we are developing a custom board based on AM335x Starter Kit, equipped with two etherent ports with physical device DP83822.

The dts file part related to ethernet configuration is the following:

    cpsw_default: cpsw_default {
       pinctrl-single,pins = <
           /* Slave 1 */
           AM33XX_IOPAD(0x90c, PIN_INPUT | MUX_MODE1) /* (H17) gmii1_crs.rmii1_crs_dv */
           AM33XX_IOPAD(0x910, PIN_INPUT | MUX_MODE1) /* (J15) gmii1_rxer.rmii1_rxer */
           AM33XX_IOPAD(0x914, PIN_OUTPUT | MUX_MODE1) /* (J16) gmii1_txen.rmii1_txen */
           AM33XX_IOPAD(0x928, PIN_OUTPUT | MUX_MODE1) /* (K17) gmii1_txd0.rmii1_txd0 */
           AM33XX_IOPAD(0x924, PIN_OUTPUT | MUX_MODE1) /* (K16) gmii1_txd1.rmii1_txd1 */
           AM33XX_IOPAD(0x940, PIN_INPUT | MUX_MODE1) /* (M16) gmii1_rxd0.rmii1_rxd0 */
           AM33XX_IOPAD(0x93c, PIN_INPUT | MUX_MODE1) /* (L15) gmii1_rxd1.rmii1_rxd1 */
           AM33XX_IOPAD(0x944, PIN_INPUT | MUX_MODE0) /* (H18) rmii1_refclk.rmii1_refclk */

           /* Slave 2 */
           AM33XX_IOPAD(0x864, PIN_INPUT | MUX_MODE3) /* (U16) gpmc_a9.rmii2_crs_dv */
           AM33XX_IOPAD(0x874, PIN_INPUT | MUX_MODE3) /* (U17) gpmc_wpn.rmii2_rxer */
           AM33XX_IOPAD(0x840, PIN_OUTPUT | MUX_MODE3) /* (R13) gpmc_a0.rmii2_txen */
           AM33XX_IOPAD(0x854, PIN_OUTPUT | MUX_MODE3) /* (V15) gpmc_a5.rmii2_txd0 */
           AM33XX_IOPAD(0x850, PIN_OUTPUT | MUX_MODE3) /* (R14) gpmc_a4.rmii2_txd1 */
           AM33XX_IOPAD(0x86c, PIN_INPUT | MUX_MODE3) /* (V17) gpmc_a11.rmii2_rxd0 */
           AM33XX_IOPAD(0x868, PIN_INPUT | MUX_MODE3) /* (T16) gpmc_a10.rmii2_rxd1 */
           AM33XX_IOPAD(0x908, PIN_INPUT | MUX_MODE1) /* (H16) gmii1_col.rmii2_refclk */
       >;
   };

    davinci_mdio_default: davinci_mdio_default {
       pinctrl-single,pins = <
           /* MDIO */
           AM33XX_IOPAD(0x948, PIN_INPUT | MUX_MODE0) /* (M17) mdio_data.mdio_data */
           AM33XX_IOPAD(0x94c, PIN_OUTPUT | MUX_MODE0) /* (M18) mdio_clk.mdio_clk */
       >;
   };

....

&mac {
   pinctrl-names = "default", "sleep";
   pinctrl-0 = <&cpsw_default>;
   pinctrl-1 = <&cpsw_sleep>;
   dual_emac = <1>;
   status = "okay";
};

&davinci_mdio {
   pinctrl-names = "default", "sleep";
   pinctrl-0 = <&davinci_mdio_default>;
   pinctrl-1 = <&davinci_mdio_sleep>;
   status = "okay";
   reset-gpios = <&gpio1 18 GPIO_ACTIVE_LOW>;
   reset-delay-us = <2>;   /* PHY datasheet states 1uS min */

   ethphy0: ethernet-phy@1 {
       reg = <1>;
   };

   ethphy1: ethernet-phy@2 {
       reg = <2>;
   };
};

&cpsw_emac0 {
   phy-handle = <&ethphy0>;
   phy-mode = "rmii";
   dual_emac_res_vlan = <1>;
};

&cpsw_emac1 {
   phy-handle = <&ethphy1>;
   phy-mode = "rmii";
   dual_emac_res_vlan = <2>;
};

&phy_sel {
   rmii-clock-ext;
};

The first port (eth0) works fine (dhcp works as well on this port), while the second (eth1) is not working.
We checked with a network sniffer that the trasmitting part (for eth1) is ok (ARP packet requests and responses are logged during a ping command), and the physical level (rmii_rxd0 and rmii_rxd1) seems ok (checked with an oscilloscope) since we see data flowing. Anyway the network communication is not working.
We have also verified that all pins connection in receiving part are ok: we configured the pins as output GPIO and checking the output on physical the level is the one expected.
We checked the DP83822 register configurations for both ports, and they are correct and and the same for both ports.

The problem has been seen and debugged on different boards, so we exclude a single faulty unit.

Here are some kernel logs at startup:

...
[    1.007968] libphy: Fixed MDIO Bus: probed
[    1.078735] davinci_mdio 4a101000.mdio: davinci mdio revision 1.6, bus freq 1000000
[    1.086779] libphy: 4a101000.mdio: probed
[    1.093311] davinci_mdio 4a101000.mdio: phy[1]: device 4a101000.mdio:01, driver TI DP83822 10/100 Mbps PHY
[    1.103566] davinci_mdio 4a101000.mdio: phy[2]: device 4a101000.mdio:02, driver TI DP83822 10/100 Mbps PHY
[    1.114605] cpsw 4a100000.ethernet: Detected MACID = 74:e1:82:8b:63:97
[    1.121777] cpsw 4a100000.ethernet: initialized cpsw ale version 1.4
[    1.128440] cpsw 4a100000.ethernet: ALE Table size 1024
[    1.134873] cpsw 4a100000.ethernet: cpsw: Detected MACID = 74:e1:82:8b:63:99
...

[    5.939999] net eth0: initializing cpsw version 1.12 (0)
[    6.043537] TI DP83822 10/100 Mbps PHY 4a101000.mdio:01: attached PHY driver [TI DP83822 10/100 Mbps PHY] (mii_bus:phy_addr=4a101000.mdio:01, irq=POLL)
...
[   15.439020] net eth1: initializing cpsw version 1.12 (0)
[   15.563550] TI DP83822 10/100 Mbps PHY 4a101000.mdio:02: attached PHY driver [TI DP83822 10/100 Mbps PHY] (mii_bus:phy_addr=4a101000.mdio:02, irq=POLL)
...

Connecting the cable to eth1 the link is correctly up:
[   18.730744] cpsw 4a100000.ethernet eth1: Link is Up - 100Mbps/Full - flow control rx/tx

We have set eth1 port to static IP configuration and run the ping command to my PC, the ifconfig command returns the following:

#ifconfig
eth0      Link encap:Ethernet HWaddr 74:E1:82:8B:63:97
          UP BROADCAST MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
          Interrupt:43

eth1      Link encap:Ethernet HWaddr 74:E1:82:8B:63:99
          inet addr:172.20.1.29 Bcast:0.0.0.0 Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B) TX bytes:576 (576.0 B)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          UP LOOPBACK RUNNING MTU:65536 Metric:1
          RX packets:11 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1232 (1.2 KiB) TX bytes:1232 (1.2 KiB)

The ethtool output for the second port:

# ethtool eth1
Settings for eth1:
   Supported ports: [ TP MII ]
   Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
   Supported pause frame use: Symmetric Receive-only
   Supports auto-negotiation: Yes
   Advertised link modes: 10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
   Advertised pause frame use: No
   Advertised auto-negotiation: Yes
   Link partner advertised link modes: 10baseT/Half 10baseT/Full
                                         100baseT/Half 100baseT/Full
   Link partner advertised pause frame use: Symmetric Receive-only
   Link partner advertised auto-negotiation: Yes
   Speed: 100Mb/s
   Duplex: Full
   Port: MII
   PHYAD: 2
   Transceiver: internal
   Auto-negotiation: on
   Supports Wake-on: d
   Wake-on: d
   Current message level: 0x00000000 (0)

   Link detected: yes

Can you please help us with this issue?

Thanks.

over 6 years ago

0 Biser Gatchev-XID over 6 years ago

TI__Guru**** 393215 points

Hi,

Please complete this checklist and post the results here: processors.wiki.ti.com/.../5x_CPSW

0 Paolo Bartolini over 6 years ago in reply to Biser Gatchev-XID

Prodigy 130 points

Here is the checklist result (some information already posted):

Only eth1 is connected to ethernet cable (eth0 works properly).

[    1.007968] libphy: Fixed MDIO Bus: probed
[    1.078735] davinci_mdio 4a101000.mdio: davinci mdio revision 1.6, bus freq 1000000
[    1.086779] libphy: 4a101000.mdio: probed
[    1.093311] davinci_mdio 4a101000.mdio: phy[1]: device 4a101000.mdio:01, driver TI DP83822 10/100 Mbps PHY
[    1.103566] davinci_mdio 4a101000.mdio: phy[2]: device 4a101000.mdio:02, driver TI DP83822 10/100 Mbps PHY
[    1.114605] cpsw 4a100000.ethernet: Detected MACID = 74:e1:82:8b:63:97
[    1.121777] cpsw 4a100000.ethernet: initialized cpsw ale version 1.4
[    1.128440] cpsw 4a100000.ethernet: ALE Table size 1024
[    1.134873] cpsw 4a100000.ethernet: cpsw: Detected MACID = 74:e1:82:8b:63:99
...
[   18.730744] cpsw 4a100000.ethernet eth1: Link is Up - 100Mbps/Full - flow control rx/tx

ethtool output:
    Is auto negotiation enabled on both ends of the link? yes
    Do the link speeds match between the TI device and the link partner? yes
    Is the Link duplex mode correct? yes

# ethtool eth1
Settings for eth1:
   Supported ports: [ TP MII ]
   Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
   Supported pause frame use: Symmetric Receive-only
   Supports auto-negotiation: Yes
   Advertised link modes: 10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
   Advertised pause frame use: No
   Advertised auto-negotiation: Yes
   Link partner advertised link modes: 10baseT/Half 10baseT/Full
                                         100baseT/Half 100baseT/Full
   Link partner advertised pause frame use: Symmetric Receive-only
   Link partner advertised auto-negotiation: Yes
   Speed: 100Mb/s
   Duplex: Full
   Port: MII
   PHYAD: 2
   Transceiver: internal
   Auto-negotiation: on
   Supports Wake-on: d
   Wake-on: d
   Current message level: 0x00000000 (0)

   Link detected: yes

# ethtool -S eth1
NIC statistics:
     Good Rx Frames: 0
     Broadcast Rx Frames: 0
     Multicast Rx Frames: 0
     Pause Rx Frames: 0
     Rx CRC Errors: 0
     Rx Align/Code Errors: 0
     Oversize Rx Frames: 0
     Rx Jabbers: 0
     Undersize (Short) Rx Frames: 0
     Rx Fragments: 0
     Rx Octets: 0
     Good Tx Frames: 9
     Broadcast Tx Frames: 9
     Multicast Tx Frames: 0
     Pause Tx Frames: 0
     Deferred Tx Frames: 0
     Collisions: 0
     Single Collision Tx Frames: 0
     Multiple Collision Tx Frames: 0
     Excessive Collisions: 0
     Late Collisions: 0
     Tx Underrun: 0
     Carrier Sense Errors: 0
     Tx Octets: 612
     Rx + Tx 64 Octet Frames: 0
     Rx + Tx 65-127 Octet Frames: 9
     Rx + Tx 128-255 Octet Frames: 0
     Rx + Tx 256-511 Octet Frames: 0
     Rx + Tx 512-1023 Octet Frames: 0
     Rx + Tx 1024-Up Octet Frames: 0
     Net Octets: 612
     Rx Start of Frame Overruns: 0
     Rx Middle of Frame Overruns: 0
     Rx DMA Overruns: 0
     Rx DMA chan 0: head_enqueue: 1
     Rx DMA chan 0: tail_enqueue: 127
     Rx DMA chan 0: pad_enqueue: 0
     Rx DMA chan 0: misqueued: 0
     Rx DMA chan 0: desc_alloc_fail: 0
     Rx DMA chan 0: pad_alloc_fail: 0
     Rx DMA chan 0: runt_receive_buf: 0
     Rx DMA chan 0: runt_transmit_bu: 0
     Rx DMA chan 0: empty_dequeue: 0
     Rx DMA chan 0: busy_dequeue: 0
     Rx DMA chan 0: good_dequeue: 0
     Rx DMA chan 0: requeue: 0
     Rx DMA chan 0: teardown_dequeue: 0
     Tx DMA chan 0: head_enqueue: 9
     Tx DMA chan 0: tail_enqueue: 0
     Tx DMA chan 0: pad_enqueue: 0
     Tx DMA chan 0: misqueued: 0
     Tx DMA chan 0: desc_alloc_fail: 0
     Tx DMA chan 0: pad_alloc_fail: 0
     Tx DMA chan 0: runt_receive_buf: 0
     Tx DMA chan 0: runt_transmit_bu: 9
     Tx DMA chan 0: empty_dequeue: 9
     Tx DMA chan 0: busy_dequeue: 0
     Tx DMA chan 0: good_dequeue: 9
     Tx DMA chan 0: requeue: 0
     Tx DMA chan 0: teardown_dequeue: 0

ifconfig output (eth1 configured with static IP):
# ifconfig
eth0      Link encap:Ethernet HWaddr 74:E1:82:8B:63:97
          UP BROADCAST MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
          Interrupt:43

eth1      Link encap:Ethernet HWaddr 74:E1:82:8B:63:99
          inet addr:172.20.1.29 Bcast:0.0.0.0 Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B) TX bytes:576 (576.0 B)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          UP LOOPBACK RUNNING MTU:65536 Metric:1
          RX packets:11 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1232 (1.2 KiB) TX bytes:1232 (1.2 KiB)

ARP packets shown in Wireshark for a ping command to my PC:
1132054   8553.577956649   TexasIns_8b:63:99   ARP   Broadcast   64   Who has 172.20.2.103? Tell 172.20.1.29 [ETHERNET FRAME CHECK SEQUENCE INCORRECT]
1132055   8553.578010044   HewlettP_60:43:23   ARP   TexasIns_8b:63:99   42   172.20.2.103 is at ac:e2:d3:60:43:23

Other requested information:
- Linux source is processor-sdk-linux-4.14.y from ti-processor-sdk-linux-am335x-evm-05.03.00.07
# uname -a
Linux buildroot 4.14.79 #1 PREEMPT Thu Aug 29 11:17:35 CEST 2019 armv7l GNU/Linux
The filesystem is built with buildroot, and has been successfully tested on AM335x Starter Kit with both eth ports working.

0 Schuyler Patton over 6 years ago in reply to Paolo Bartolini

TI__Mastermind 40130 points

Hi,

Thank you for requested information and only hooking eth1 to debug the interface. The ethtool -S is looking at the HW statistics of the CPSW interface.

Good Rx Frames: 0
Broadcast Rx Frames: 0
Multicast Rx Frames: 0
Pause Rx Frames: 0
Rx CRC Errors: 0
Rx Align/Code Errors: 0
Oversize Rx Frames: 0
Rx Jabbers: 0
Undersize (Short) Rx Frames: 0
Rx Fragments: 0
Rx Octets: 0
Good Tx Frames: 9
Broadcast Tx Frames: 9
Multicast Tx Frames: 0

The link is detected on eth1, you are seeing the broadcast packets from eth1 and the hardware statistics are showing packets leaving the interface. This indicates that the TX path is good. The RX path however is not, something is going wrong here. The sniffer is showing the ARP responses are being sent to the AM335x but they are not been seen at the hardware level. This leaves between the MAC and the connector assuming a direct connection between the PC and the eth1 port you are looking at. The pin mux and DT entries look good but no packets on the RX side of things.

I think the next step is HW level looking at the RX path in from the connector. Does a scope show any traffic on the RX looking at the RX lines to see if there is any traffic on them?

Best Regards,

Schuyler

0 Paolo Bartolini over 6 years ago in reply to Schuyler Patton

Prodigy 130 points

Hi, maybe we found the solution.

Looking at http://www.ti.com/lit/ds/symlink/am3358.pdf

(10) Silicon revision 1.0 devices only provide the MMC2_DAT7 signal when Mode3 is selected. Silicon revision 2.0 and newer devices implement another level of pin multiplexing which
provides the original MMC2_DAT7 signal or RMII2_CRS_DV signal when Mode3 is selected. This new level of of pin multiplexing is selected with bit zero of the SMA2 register. For more
details refer to Section 1.2 of the AM335x Technical Reference Manual.

Our silicon revision is AM3354B (2.1).

When we read the SMA2 register (in u-boot for example) the value returned is 0x0.
Setting it to 0x1 and booting the kernel, eth1 starts working.

In you opinion, is this solution we found correct? Does this setting make sense?
Is there a better way to do this configuration (maybe pin-mux)?

Thanks,
regards.

Paolo

0 Schuyler Patton over 6 years ago in reply to Paolo Bartolini

TI__Mastermind 40130 points

Hi,

I am out of the office currently, I will return on the 4th. Could you please attach a schematic snippet showing the connection between the CPSW and the PHY?

Best Regards,

Schuyler

0 Paolo Bartolini over 6 years ago in reply to Schuyler Patton

Prodigy 130 points

Hi,
I would prefer to send the schematic snippet directly to you, without sharing it publicly, as it is a company document.

Is this possible?
If so, could you please provide a way to send you the document?

Thanks,
Regards

Paolo

0 Schuyler Patton over 6 years ago in reply to Paolo Bartolini

TI__Mastermind 40130 points

Hi,

I recognize potential sensitivity of sharing the complete schematic. I really don't need the entire schematic, for this I only need a small portion showing the network path between the SOC and PHY's. Is the network isolated to a page?

Best Regards,

Schuyler

0 Paolo Bartolini over 6 years ago in reply to Schuyler Patton

Prodigy 130 points

Hi,
I think that there is no need to share the schematic, as the problem is now solved and eth1 works fine.
Anyway, I would like to have a confirmation that the solution we found is correct and makes sense.

The bit 0 in SMA2 register (1320h) is set 1 (Select RMII2_CRS_DV on GPMC_A9 pin in MODE3) see (10) at page 48 of www.ti.com/.../am3358.pdf .

Can you plese comment?

Thanks a lot,
regards

Paolo

0 Schuyler Patton over 6 years ago in reply to Paolo Bartolini

TI__Mastermind 40130 points

Hi,

I believe you are correct, I am verifying with the developer to see if that is the case. To me at least it looks like eth1 essentially has to have this bit set. Since you found the bug you can post the patch to the kernel mailing list for credit for resolving the issue. If you choose to do that please attach the post here as well.

I will post to this thread when I hear back from the developer.

Thanks and Best Regards,

Schuyler

0 Schuyler Patton over 5 years ago in reply to Schuyler Patton

TI__Mastermind 40130 points

Hi,

You solution is the correct thing to do for the I/O set that you have chosen to use. I am not sure where you applied the patch but the one TI recommends is setting the SMA register in U-boot and letting the kernel rely on the inheritance of the setting.

This use case is using the 3rd I/O set defined for RMII2, if the first 2 I/O sets are used this SMA register does not matter since a different pin muxes are being used.

Regards,

Schuyler

Processors

Processors forum

AM3354: Eth1 not working