PROCESSOR-SDK-AM437X: [AM437x] Ping fails on eth0 and eth1 on custom hardware

Mahidar,

I just received a good suggestion from Schuyler Patton. He pointed out that you're missing this:

&phy_sel {
        rmii-clock-ext = <1>;
};

Please try that out to see if it resolves your issue.

Best regards,
Brad

Hi Brad,

 With the suggested change also we donot see PING working for us.

Regards,

Mahidhar.

That was definitely an important change, so that should be kept permanently regardless.

Can you verify that a 50 MHz clock is present?  Is it being generated by the PHY or do you have a separate crystal oscillator providing the 50 MHz clock to both the processor and the PHY?

Yes the 50Mhz clock is present. The clock is being provided by separate 50MHz crystal oscillator for the PHY’s.

Also everything looks good from hardware perspective, voltage levels are @3.3V, signal integrity looks good. 

Hi,

- The boot log is showing the PHY driver is identified and PHYs are at the correct address. 

- The ethtool is showing a link established, but the ethtool statistics is showing no received packets. It is also showing no received errors either.

I would like to suggest the following debug steps:

- The tx packets from device to device need to be followed and proved that that they are showing up at the link partner. While the statistics are showing packets leaving (TX count increasing) we need to see that they showed up on the connected PC. Using Wireshark here on the laptop here would be ideal because you can see the types of packets being sent from the board. You can also test this without using Wireshark too. For this test direct connect the board using eth0 to the laptop and please leave eth1 disconnected. 

- Before issuing the ping command please perform the following commands you were doing earlier in the post ifconfig -a, ethtool eth0, ethtool -S eth0, please perform these same commands on the laptop too. If the laptop is not running linux you will need to use wireshark.

- Run the ping command and after a few seconds stop the command and repeat the previous steps for ifconfig -a ethtool etc. for both sides of the link partner. 

With the results we want to see if the RX packet count on the laptop is showing an increase. If the RX count on the laptop is increasing that indicates the packets from the board are being received.  Please attach (not cut and paste) the results of the commands to this thread. If you use wireshark please attach the capture file as well.

You can also repeat the test with eth1 but leave eth0 disconnected.

Best Regards,

Schuyler

Hi Schuyler,

We are able to see arp broadcast packets while performing ping from device to PC on wireshark but still PING isnt successful and seems there is no RX activity. 

Device to PC <Wireshark capture>:

2 0.204155 TexasIns_94:c1:a6 Broadcast ARP 64 Who has 169.254.1.60? Tell 169.254.1.1

3 0.204321 GoodWayI_16:56:67 TexasIns_94:c1:a6 ARP 42 169.254.1.60 is at 00:50:b6:16:56:67

4 1.264935 TexasIns_94:c1:a6 Broadcast ARP 64 Who has 169.254.1.60? Tell 169.254.1.1

5 1.265165 GoodWayI_16:56:67 TexasIns_94:c1:a6 ARP 42 169.254.1.60 is at 00:50:b6:16:56:67

While performing PC to device ping operation we didnt see anything on tcpdump. 

PC to device <Wireshark capture>:

296 2147.321328 GoodWayI_16:56:67 Broadcast ARP 42 Who has 169.254.1.1? Tell 169.254.1.60
301 2208.321803 GoodWayI_16:56:67 Broadcast ARP 42 Who has 169.254.1.1? Tell 169.254.1.60

Regards,

Mahidhar

Nikhilesh,

Thanks for your patience.  The severe winter storms in Texas are impacting our ability to answer questions.

I'm going to do some further digging to try and come up with suggestions.  Hopefully Schuyler Patton will be back online soon.

Best regards,
Brad

At run-time can you please read the CTRL_GMII_SEL register at address 0x44e10650?

devmem2 0x44e10650

There are a few critical values in that register I'd like to double check.

Thanks,
Brad

Hi Brad

Brad Griffis said:

At run-time can you please read the CTRL_GMII_SEL register at address 0x44e10650?

devmem2 0x44e10650

There are a few critical values in that register I'd like to double check.

root@Node1442fc94c1a6:~# ./devmem2 0x44e10650
/dev/mem opened.
Memory mapped at address 0xb6f01000.
Read at address 0x44E10650 (0xb6f01650): 0x000000C5

GMII sel is  set to RMII mode.

RMII1_IO_CLK_EN is set to Enable RMII clock to be sourced from chip pin

Regards,

Mahidhar.

Hello Brad, 

In CTRL_GMII_SEL register the possible values for RMIIx_IO_CLK_EN are:

1 : Enable RMII clock to be sourced from chip pin
0 : Enable RMII clock to be sourced from PLL

Our understanding is:

1 : Enable RMII clock to be sourced from chip pin - This means clock is sourced externally - which is currently set - please let us know if our undersetting is wrong 
0 : Enable RMII clock to be sourced from PLL - This means clock is sourced from PLL

I looked in to the dts/dtb file and can see we have added

 &phy_sel {
        rmii-clock-ext = <1>;
};

Nikhilesh,

Oh geez.  Sorry, I totally reversed that.  You're right.  I was interpreting "sourced from chip pin" to mean "originating from chip pin" or "generated at chip pin".

Sorry for any confusion.  I'll edit my previous post!  (And get another cup of coffee!)

Best regards,
Brad

To add to the observations: 

• The refclk looks stable
• We probed the Rx and Tx lines between PHY and MAC. We see the RX data between PHY and MAC. 
• We see Rx data between PHY and Mac we do not see Rx interrupt counters increment at all on micro side 
• As indicated before if we ping from Device -> PC, on wireshark we are able to see ARP packets, we see Rx signal between PHY and MAC but "tcpdump" on device does not show anything  
• We also looked at clock signal going to PHY and MAC, the clocks look good, we did not see any clock skewing.  

Hi,

You have traced that the ARP process is transmitted correctly and the response is coming back to your board. Since the MAC hw statistics Good Rx Frames is showing 0 this could indicate something on the RX pin mux needs to be verified. 

Good Rx Frames: 0

Please attach a snippet of the schematic and the DTS snippet for the CPSW interface.

Best Regards,

Schuyler

Hi Schuyler,

DTS  snippet of CPSW interface.

cpsw1_default: cpsw1_default {
pinctrl-single,pins = <
/* Slave 1 */
AM4372_IOPAD(0x90c, PIN_INPUT | MUX_MODE1) /* (B14) mii1_crs.rmii1_crs_dv */
AM4372_IOPAD(0x910, PIN_INPUT | MUX_MODE1) /* (B13) mii1_rx_er.rmii1_rxer */
AM4372_IOPAD(0x914, PIN_OUTPUT | MUX_MODE1) /* (A13) mii1_tx_en.rmii1_txen */
AM4372_IOPAD(0x928, PIN_OUTPUT | MUX_MODE1) /* (B15) mii1_txd0.rmii1_txd0 */
AM4372_IOPAD(0x924, PIN_OUTPUT | MUX_MODE1) /* (A14) mii1_txd1.rmii1_txd1 */
AM4372_IOPAD(0x940, PIN_INPUT | MUX_MODE1) /* (F17) mii1_rxd0.rmii1_rxd0 */
AM4372_IOPAD(0x93c, PIN_INPUT | MUX_MODE1) /* (B16) mii1_rxd1.rmii1_rxd1 */
AM4372_IOPAD(0x944, PIN_INPUT | MUX_MODE0) /* (A16) rmii1_ref_clk.rmii1_refclk */
>;
};

cpsw2_default: cpsw2_default {
pinctrl-single,pins = <
/* Slave 2*/
AM4372_IOPAD(0x864, PIN_INPUT | MUX_MODE8) /* (B4) gpmc_a9.rmii2_crs_dv */
AM4372_IOPAD(0x874, PIN_INPUT | MUX_MODE3) /* (B3) gpmc_wpn.rmii2_rxer */
AM4372_IOPAD(0x840, PIN_OUTPUT | MUX_MODE3) /* (C3) gpmc_a0.rmii2_txen */
AM4372_IOPAD(0x854, PIN_OUTPUT | MUX_MODE3) /* (E7) gpmc_a5.rmii2_txd0 */
AM4372_IOPAD(0x850, PIN_OUTPUT | MUX_MODE3) /* (D7) gpmc_a4.rmii2_txd1 */
AM4372_IOPAD(0x86c, PIN_INPUT | MUX_MODE3) /* (D8) gpmc_a11.rmii2_rxd0 */
AM4372_IOPAD(0x868, PIN_INPUT | MUX_MODE3) /* (G8) gpmc_a10.rmii2_rxd1 */
AM4372_IOPAD(0x908, PIN_INPUT | MUX_MODE1) /* (D16) mii1_col.rmii2_refclk */

>;
};

Schematic snippet we will share it offline separately.

Regards,

Maheedhar.

Hi,

Thank you for dts and additional data. Pin mux wise this looks correct. I need to discuss with additional team members. I will respond on Monday.

I apologize for repeating this but until this statistic is non-zero the ethtool -S eth0 output then are no packets are being received.

Best Regards,

Schuyler

Hello, 

I was looking in to some references board device tree files, where RMII is used. I came across AM43x EPOS EVM (am43x-epos-evm.dts), for this EVM I see the RXDV pin configured in dts for RMII pin configuration, But when we looked at the board schematic the RXDV pin is not connected any where (not used). 
I also checked RMII vs MII signals, as per PHY data sheet RMII doesn't use RXDV signal. 

am43x-epos-evm.dts RMII pin config:

cpsw_default: cpsw_default {
pinctrl-single,pins = <
/* Slave 1 */
AM4372_IOPAD(0x90c, PIN_INPUT_PULLDOWN | MUX_MODE1) /* mii1_crs.rmii1_crs */
AM4372_IOPAD(0x910, PIN_INPUT_PULLDOWN | MUX_MODE1) /* mii1_rxerr.rmii1_rxerr */
AM4372_IOPAD(0x914, PIN_OUTPUT_PULLDOWN | MUX_MODE1) /* mii1_txen.rmii1_txen */
AM4372_IOPAD(0x918, PIN_INPUT_PULLDOWN | MUX_MODE1) /* mii1_rxdv.rmii1_rxdv */
AM4372_IOPAD(0x924, PIN_OUTPUT_PULLDOWN | MUX_MODE1) /* mii1_txd1.rmii1_txd1 */
AM4372_IOPAD(0x928, PIN_OUTPUT_PULLDOWN | MUX_MODE1) /* mii1_txd0.rmii1_txd0 */
AM4372_IOPAD(0x93c, PIN_INPUT_PULLDOWN | MUX_MODE1) /* mii1_rxd1.rmii1_rxd1 */
AM4372_IOPAD(0x940, PIN_INPUT_PULLDOWN | MUX_MODE1) /* mii1_rxd0.rmii1_rxd0 */
AM4372_IOPAD(0x944, PIN_INPUT_PULLDOWN | MUX_MODE0) /* rmii1_refclk.rmii1_refclk */
>;
};

I am not sure if there is a clue here, Is the driver somehow looking for this pin to be configured? 

NIKHILESH CHAVAN said:

AM4372_IOPAD(0x918, PIN_INPUT_PULLDOWN | MUX_MODE1) /* mii1_rxdv.rmii1_rxdv */

That particular signal / mode of operation is non-existent as far as I can tell.  I'm checking with some others for verification.  I don't think it has any relationship to RMII.

I noticed that all of the pins in the EVM have the internal pulldown activated.  I'm not aware of any such requirement, but it would be an easy test for you to try in the interim.  Will you please try that out?  I don't think it should make a difference, but given that we still haven't resolved the issue, it's a good test.

Hello Brad, 

We have already tried configuring the pins with internal pulldown activated, It does not make any difference. 

We have also tried configuring only cpsw_emac0 just to see if we are facing any problems due to dual_emac, but that also did not make nay difference. 

Thanks for the update.  I'm glad to hear it's working now!