• State Resolved
  • Locked Locked
  • Replies 20 replies
  • Subscribers 103 subscribers
  • Views 12022 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM335x RGMII Problem (PHY KSZ9021RL): ping fails

Davi Evangelista
Expert 1090 points

Hello,

I have a custom PCB that has an AM3552 microprocessor and I'm having some trouble trying to make my ethernet bus work properly. I'm using an external PHY, which is identified when I bring the interface up, but so far I wasn't able to ping an outside IP (localhost works just fine).

The board uses the pins reserved for rgmii1 to other purposes, leaving rgmii2 to serve as the ethernet interface. Our PHY transceiver is the KSZ9021RL from Micrel.

First I configured the mux in the configuration file and it looks like this:

/* Module pin mux for rgmii2 */
static struct pinmux_config rgmii2_pin_mux[] = {
   {"mdio_clk.mdio_clk",   OMAP_MUX_MODE0 | AM33XX_PIN_OUTPUT_PULLUP},
   {"mdio_data.mdio_data", OMAP_MUX_MODE0 | AM33XX_PIN_INPUT_PULLUP},
   {"gpmc_a0.rgmii2_tctl", OMAP_MUX_MODE2 | AM33XX_PIN_OUTPUT},
   {"gpmc_a1.rgmii2_rctl", OMAP_MUX_MODE2 | AM33XX_PIN_INPUT_PULLDOWN},
   {"gpmc_a6.rgmii2_tclk", OMAP_MUX_MODE2 | AM33XX_PIN_OUTPUT},
   {"gpmc_a7.rgmii2_rclk", OMAP_MUX_MODE2 | AM33XX_PIN_INPUT_PULLDOWN},
   {"gpmc_a2.rgmii2_td3",  OMAP_MUX_MODE2 | AM33XX_PIN_OUTPUT},
   {"gpmc_a3.rgmii2_td2",  OMAP_MUX_MODE2 | AM33XX_PIN_OUTPUT},
   {"gpmc_a4.rgmii2_td1",  OMAP_MUX_MODE2 | AM33XX_PIN_OUTPUT},
   {"gpmc_a5.rgmii2_td0",  OMAP_MUX_MODE2 | AM33XX_PIN_OUTPUT},
   {"gpmc_a8.rgmii2_rd3",  OMAP_MUX_MODE2 | AM33XX_PIN_INPUT_PULLDOWN},
   {"gpmc_a9.rgmii2_rd2",  OMAP_MUX_MODE2 | AM33XX_PIN_INPUT_PULLDOWN},
   {"gpmc_a10.rgmii2_rd1", OMAP_MUX_MODE2 | AM33XX_PIN_INPUT_PULLDOWN},
   {"gpmc_a11.rgmii2_rd0", OMAP_MUX_MODE2 | AM33XX_PIN_INPUT_PULLDOWN},
   {NULL, 0},
};

and when I set it up I also call

am33xx_cpsw_init(AM33XX_CPSW_MODE_RGMII, NULL, "0:01");

to configure it to RGMII.

On the console, I can bring the interface up and list it with ethtool:

root@UCC3:~# ifconfig eth1 up
[ 58.139617] net eth1: CPSW phy found : id is : 0x221611
[ 60.138305] PHY: 0:01 - Link is Up - 100/Full
root@UCC3:~# ifconfig eth1 192.168.100.15
root@UCC3:~# ifconfig eth1
eth1   Link encap:Ethernet HWaddr BC:6A:29:96:AC:ED
       inet addr:192.168.100.15 Bcast:192.168.100.255 Mask:255.255.255.0
       UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
       RX packets:0 errors:0 dropped:0 overruns:0 frame:0
       TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
       collisions:0 txqueuelen:1000
       RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
root@UCC3:~# ethtool eth1
Settings for eth1:
     Supported ports: [ TP AUI BNC MII FIBRE ]
     Supported link modes: 10baseT/Half 10baseT/Full
                           100baseT/Half 100baseT/Full
                           1000baseT/Half 1000baseT/Full
     Supports auto-negotiation: Yes
     Advertised link modes: 10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Half 1000baseT/Full
     Advertised pause frame use: No
     Advertised auto-negotiation: Yes
     Speed: 100Mb/s
     Duplex: Full
     Port: MII
     PHYAD: 1
     Transceiver: external
     Auto-negotiation: on
     Current message level: 0x00000000 (0)

     Link detected: yes

But I still can't ping anything. When I do the same thing on the console of our Beaglebone the ping command works as expected.

I read a lot of topics in the forum, but so far none of them was of much use in my case. In any case, we boot the board via NAND and will not use the ETH for it.

I'm trying to check if there is some specific configuration for this particular PHY in the Kernel that would need to be adjusted. Anything regarding this topic would be of much help!

So, that is pretty much all the information I think is relevant. Thank you for the patience to read all this!

  • 0
    TI__Mastermind 27890 points

    Hello Davi,

    Did you perform a timing analysis on this RGMII interface prior to committing the design to PCB? 

  • 0 in reply to -DK-
    Expert 1090 points

    We haven't decided yet which PHY will be present in the final PCB. All the rgmii2 pins goes to a header and we connect an evaluation board designed to test the KSZ9021 PHY to that header.

    With this design, we did check the MDIO CLK (MDC) and both the TXCLK (TXC) and RXCLK (RXC): all 3 pins are ok.  TXC and RXC are operating in 25MHz and MDC in 1MHz.

    Furthermore, ethtool is able to identify whether the RJ45 cable is connected or not. Doesn't that mean that the MPU is able to communicate with the PHY as it shoud?

  • 0 in reply to Davi Evangelista
    Expert 1090 points

    Turns out I missed the Kernel configuration that enabled the PHY device driver for Micrel, but unfortunately that alone didn't solve my problem...

    It did change the output of ethtool though, as you can see below. 

    root@UCC3:~# ethtool eth1
    Settings for eth1:
         Supported ports: [ TP MII ]
         Supported link modes: 10baseT/Half 10baseT/Full 
                               100baseT/Half 100baseT/Full 
         Supports auto-negotiation: Yes
         Advertised link modes: 10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
         Advertised pause frame use: Symmetric
         Advertised auto-negotiation: Yes
         Speed: 100Mb/s
         Duplex: Full
         Port: MII
         PHYAD: 1
         Transceiver: external
         Auto-negotiation: on
         Current message level: 0x00000000 (0)

         Link detected: yes

    There is another thing that I don't understand: dmesg doesn't print the module description as a normal driver does when it is called. The only messages regarding the ethernet are:

    [ 14.658347] net eth1: CPSW phy found : id is : 0x221611
    [ 17.649236] PHY: 0:01 - Link is Up - 100/Full

  • 0 in reply to Davi Evangelista
    Prodigy 230 points

    Check the phy skew values. We had to adjust ours (different phy, but I think this applies to all GB phys). You may be missing the data eye on either the rx or tx.

    Can you wireshark the network when you ping to see if the packet makes it out and or back? That should help narrow it down.

    cheers

    /Tim

  • 0 in reply to Tim Iskander
    TI__Mastermind 27890 points

    This is good advice and ties in with my question about the timing analysis.

    The RGMII interface timing is very tight so care must be taken to ensure that the relevant timing parameters of the selected PHY, our MAC, and the board design itself are accounted for prior to committing to PCB. As Tim points out above, many PHYs have registers that allow SW to adjust skew internally which can help close timing on designs that are marginal or slightly out of spec. This interface timing can be calculated, but since you already have boards, it would be easier to just scope the interface to see what your timing looks like and adjust the PHY skew accordingly.

    As for your question regarding the link status, this information is (typically) provided by the PHY via the MDIO bus, not via the RGMII interface itself. 

  • 0 in reply to -DK-
    Expert 1090 points

    Update: with the oscilloscope we ware able to identify a huge noise in our design coming from the ground and that is possibly the reason for this misbehave. For the time being there is little we can do to solve this problem (due to technical reasons), so this matter will remain unclear until we solve this issue.

    Anyhow, thank you for the support!

  • 0 in reply to Davi Evangelista
    Prodigy 50 points

    Hi

    KSZ9021 has three RGMII timing registers(104,105,106) and we  need to configure these registers, try with F0F0 in 104 and 0000 in 105 and 0000 in 106 registers once.

    Shiva

  • 0 in reply to shiva raju
    Expert 1090 points

    Hello again. It has been quite some time, but now I have a board that does not show any noise that would mess up my analysis.

    Turns out Micrel's driver in the Kernel was outdated and I had to update it with the version available in the vendor's website. Regardless, the scenario hasn't changed much.

    First things first, I still use the same pinmux configuration as shown in my first post and I still call the line below in the board configuration file:

    am33xx_cpsw_init(AM33XX_CPSW_MODE_RGMII, NULL, NULL);

    I haven't changed the registers yet (as shiva raju proposed) because it seams I'm experiencing an unrelated problem, which I'm explaining below. Once I solve this issue I can try that!

    After booting the board I try to bring the interface up and set it to 100Mb/s. These are the messages I get:

    # ifconfig eth0 up
    [   19.948221] net eth0: CPSW phy found : id is : 0x221611
    [   28.942199] PHY: 0:01 - Link is Up - 1000/Full
    # ethtool -s eth0 speed 100 duplex full
    [  35.942431] PHY: 0:01 - Link is Down
    [  37.942341] PHY: 0:01 - Link is Up - 100/Full
    # ethtool eth0
    Settings for eth0:
            Supported ports: [ TP MII ]
            Supported link modes:   10baseT/Half 10baseT/Full
                                    100baseT/Half 100baseT/Full
                                    1000baseT/Half 1000baseT/Full
            Supported pause frame use: Symmetric Receive-only
            Supports auto-negotiation: Yes
            Advertised link modes:  100baseT/Full
            Advertised pause frame use: No
            Advertised auto-negotiation: Yes
            Speed: 100Mb/s
            Duplex: Full
            Port: MII
            PHYAD: 1
            Transceiver: external
            Auto-negotiation: on
            Current message level: 0x00000000 (0)

            Link detected: yes

    By now both the PHY and the PC I connect the board to reports that the link is up, but when I configure their IP and try to ping one another I'm told that the network is unreachable and I get 100% packet loss on both sides.

    To debug what is wrong I'm using a scope. I can see that TX_CLK is not 25MHz as expected, it's 10MHz. RX_CLK, on the other hand, behaves normally (@25MHz). If I use ethtool to lower the connection's speed to 10Mb/s, TX_CLK goes to 1Mhz and RX_CLK goes to 2.5MHz. There is clearly a problem here...

    I'm in contact with Micrel's support to see whether this is related to their driver, but it would be nice to be sure that I'm doing everything right in the platform side of things...

    So, does anyone knows anything that would make TX_CLK behaves like this?

    Thanks.


    DAVI

  • 0 in reply to Davi Evangelista
    Expert 1090 points

    Just an update to my previous post.

    The problem with the clock frequency was related with the processor's PLL configuration and is no longer a concern. Both TX_CLK and RX_CLK are @25MHz for 100Mb/s and @2.5MHz for 10Mb/s.

    That alone didn't solve the communication's problem though... Any ideas would be highly appreciated!

    DAVI

  • 0 in reply to Davi Evangelista
    TI__Mastermind 27890 points

    Could you provide the contents of the relevant AM335x CPSW_SL_MACCTRL and CPSW_STATS registers while the device is attempting PING?

  • 0 in reply to -DK-
    Expert 1090 points

    How can I read these registers? Also, I haven't found them neither on the Technical Reference Manual nor on the code...

    DAVI

  • 0 in reply to Davi Evangelista
    TI__Mastermind 27890 points

    Sorry...I pulled the register name from another device...same function, different name. That doesn't help your search. ;)

    CPSW_STATS @  TRM section 14.5.4

    MACCONTROL @ TRM section 14.5.7

  • 0 in reply to -DK-
    Expert 1090 points

    I'm attaching the output of the test.

    I had to modify the file drivers/net/ethernet/ti/cpsw.c in order to print all the information once I called the sysfs interface, now all registers are there. The RX packets received all comes from the PC I connect my board to, which was also attempting to ping. I also configured the bus to 10Mb/s prior to the content in the file attached.

    As the scope shows, there is not a single TX packet being sent to the PHY, but RX is apparently fine (signal-wise)...

    DAVI

    4530.pingtest.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    root@UCC3:~# cat /sys/devices/platform/omap/cpsw/net/eth0/hw_stats
    CPSW Statistics:
    rxgoodframes ............................ 0
    rxbroadcastframes ....................... 0
    rxmulticastframes ....................... 0
    rxpauseframes ........................... 0
    rxcrcerrors ............................. 0
    rxaligncodeerrors ....................... 0
    rxoversizedframes ....................... 0
    rxjabberframes .......................... 0
    rxundersizedframes ...................... 0
    rxfragments ............................. 0
    rxoctets ................................ 0
    txgoodframes ............................ 0
    txbroadcastframes ....................... 0
    txmulticastframes ....................... 0
    txpauseframes ........................... 0
    txdeferredframes ........................ 0
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

  • 0 in reply to Davi Evangelista
    Expert 1285 points

    Davi,

    I am working a AM335x custom board with AR8035 Gigabit PHY. My TX_CLK is 10MHz for 100M connection. May I know how you you workaround this?

    -Eric

  • 0 in reply to Eric Lee4
    Expert 1285 points

    Davi,

    My RX_CLK is 25MHz that is correct for 100M. This is exactly the same situation that you met before. BTW, My TX_CLK is 50MHz for giga connection that is supposed to be 125Mhz. You said it can be fixed by configuring CPU's PLL. Could you share me how? Much thanks.

    -Eric

  • 0 in reply to Eric Lee4
    Expert 1090 points

    The PLL configuration is one of the first things the SPL does. Look up for pll_init() in U-Boot-2013.01.01-amsdk-06.00.00/arch/arm/cpu/armv7/am33xx/clock.c and you should have a better idea. It's also mandatory to respect the hardware set up for the OPP mode you selected in order to get the correct PLL, The default behaviour is to send an I2C message to the PMIC beforehand.

    DAVI

  • 0 in reply to Eric Lee4
    TI__Mastermind 27890 points

    Eric Lee4 said:

    Davi,

    My RX_CLK is 25MHz that is correct for 100M. This is exactly the same situation that you met before. BTW, My TX_CLK is 50MHz for giga connection that is supposed to be 125Mhz. You said it can be fixed by configuring CPU's PLL. Could you share me how? Much thanks.

    -Eric

    Hi Eric,

    RX_CLK is provided by the PHY, not the MAC. In your case it seems that the PHY is providing the correct clock, but the MAC is not.  Please take a look at this post:

    http://e2e.ti.com/support/arm/sitara_arm/f/791/p/366976/1304323.aspx#1304323

    If your TX_CLK for RGMII-1G is 50MHz, I think you will find your CORE_PLL settings are incorrect.

  • 0 in reply to -DK-
    Expert 1285 points

    DK,

    After checking my PLL settings, I can have two RGMII ports working correctly. Thanks.

    -Eric

  • 0 in reply to Eric Lee4
    TI__Mastermind 27890 points

    Eric Lee4 said:

    DK,

    After checking my PLL settings, I can have two RGMII ports working correctly. Thanks.

    -Eric

    Thats great new Eric, glad I could help.

    Could you please verify my answer by clicking the "Verify Answer" button on my post? That will close the thread.

  • 0 in reply to -DK-
    Expert 1090 points

    Actually, the thread is mine and I never got the ping to work with that design... In the end we changed from an custom external ETH board to an USB-ETH interface, but the main issue never got solved, probably due a hardware problem in our side.

    DAVI