DP83869HM: DP83869HM + LAN7801: No IP assigned under Linux "5.6.19-v7+ #10 SMP" armhf

Hi Michal,

I looked at the phy_5F00_regs.txt file and I see that register 0xC01 is 0x6149. This showsthat the link is not up and the PHY will not be able to receive data. We can start off with some basic checks to find out why the link is not up.

What is the link partner connected to DP83869 over fiber cable? Does the link partner work with other fiber Ethernet devices?

Can you also confirm that the SFP transceivers used on DP83869 and link partners are the same wavelength and that the fiber cable used is the correct one?

In you DP83869 design, are you using the signal detect pin of the SFP transceiver as input to the PHY?

-Regards

Aniruddha

Hi Aniruddha,

Aniruddha Khadye said:

I looked at the phy_5F00_regs.txt file and I see that register 0xC01 is 0x6149. This showsthat the link is not up and the PHY will not be able to receive data. We can start off with some basic checks to find out why the link is not up.

Great!

Aniruddha Khadye said:

What is the link partner connected to DP83869 over fiber cable? Does the link partner work with other fiber Ethernet devices?

The link partner is a Mikrotik router with SFP module. The router works with other fiber ethernet devices. SFP interface LED on the router is on with not-working network card connected to the router.

Aniruddha Khadye said:

Can you also confirm that the SFP transceivers used on DP83869 and link partners are the same wavelength and that the fiber cable used is the correct one?

I can't confirm that (working on it) but I can confirm this setup works with different network card usign the current cable and the current optical transceivers. Is this verification sufficient?

Aniruddha Khadye said:

In you DP83869 design, are you using the signal detect pin of the SFP transceiver as input to the PHY?

Yes. We are now verifying the logic levels on the signal detect pin.

Best wishes,
Michal.

Hi Michal,

Thanks for the additional data. When you say that the setup works with a different network card, it might be possible that the other network card uses a different SFP transceiver than the Mikrotik router.While you are checking the details about the Mikrotik SFP transceiver, can you check if it matches to the transceivers used on the other network cards that work?

-Regards

Aniruddha

Hi Aniruddha,

Aniruddha Khadye said:

When you say that the setup works with a different network card, it might be possible that the other network card uses a different SFP transceiver than the Mikrotik router.While you are checking the details about the Mikrotik SFP transceiver, can you check if it matches to the transceivers used on the other network cards that work?

Sure, I confirm, that both transceivers are exactly the same type and the optical cable also matches.

We continue to investigate the signal detect pin.

Is there anything besides the signal detect we should check right now?

Best wishes,

Michal.

Hi Aniruddha,

I confirm the signal_detect pin works as expected and is being used in the hardware design.

We have replicated our hardware setup and now I confirm I've came to consistent results among the hardware setups. Optical link is up but linux reports the link is down.

I attach updated register dump.

/cfs-file/__key/communityserver-discussions-components-files/138/phy_5F00_regs_2D00_link_5F00_up.txt

I also attach $ ip a command's output.

$ ip a
...
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether 00:80:0f:78:03:cd brd ff:ff:ff:ff:ff:ff

Best regards,
Michal.

Hi Aniruddha,

is there anything new on this topic? Is there a chance I'll get some reply that will help solve the problem within one week from now?

Best regards,

Michal.

HI Michal,

Sorry for the delayed reply. From the register dump we can confirm that the link is up. Register 0x01 bit 2 is '1' which indicates the link is up. I am not sure why then linux doesn't register it as link up. Usually the MAC will read register 0x01 to see if the link has been established and then indicate if its UP or DOWN. Debugging higher layer Linux code would be out of scope for this forum but I can recommend one small check. Can you see if the interface has been disabled by the 'ipconfig' command?

-Regards

Aniruddha

Hi Aniruddha,

thanks for your reply. I can't use ipconfig since it is not present on embedded target, but I've tried to enable the interface via ifconfig and it did not help.

"Link up" status described in reply from Aug 13, 2020 6:16 PM was achieved by restarting autonegotiation as suggested in application note https://www.ti.com/lit/pdf/snla305 . Unfortunatelly this leads to following state:

1. The phy reports "Link is up" in register dump (attached in the post Aug 13, 2020 6:16 PM ).
2. Target Linux reports "Link is down".
3. No data communication via sfp interface among target Linux and other devices on the local network is possible.
4. Sometimes the "SFP interface LED" on the router turns off.

Without "autonegotiation restart" both the target Linux and PHY registers always report the link is down.

I understand that higher level layer debugging may be necessary.

Questions:

1. Is there any known case with the same link setup (optical link, 1 Gbps, Linux with the same drivers as attached before) that works?
2. Is there anything to avoid extra auto-negotiation restart to make the PHY recognize the link and report "Link is up" in BMSR register (bit 2) ?

Best regards,

Michal.

Hi Michal,

How are you configuring the PHY to enter into Fiber mode? Are you using boot strap resistors to directly enter into Fiber mode at start-up or do you start-up in a different mode and activate Fiber mode through registers. If you are activating fiber mode through register, can you try adding a soft restart instruction at the end of your Fiber mode initialization script. Soft restart can be done by writing 0x4000 to register 0x1F.

Regarding your Q1, we have tested our own boards with linux at the time of the linux driver development. 

For Q2, BMSR register bit 2 is Latch Low bit. So when it transitions from low to high it needs to be read twice. Is it possible to change your link recognition algorithm to read the register twice?  This might also explain why when you read the registers it shows Link up but Linux target report link down

-Regards

Aniruddha

Hi Aniruddha,

Aniruddha Khadye said:

How are you configuring the PHY to enter into Fiber mode?

We use both configuration methods:

• Via strapping pins:
  • RGMII-to-1000BASE-X
  • Signal detect is being used
  • Fiber auto negotiation is enabled
• Via software, TI Linux driver, function "dp83869_configure_mode":

static int dp83869_configure_mode(struct phy_device *phydev,
                  struct dp83869_private *dp83869)
{
...
    /* Below init sequence for each operational mode is defined in
     * section 9.4.8 of the datasheet.
     */
    ret = phy_write_mmd(phydev, DP83869_DEVADDR, DP83869_OP_MODE,
                dp83869->mode); /* dp83869->mode == DP83869_RGMII_1000_BASE */
    if (ret)
        return ret;

    ret = phy_write(phydev, MII_BMCR, MII_DP83869_BMCR_DEFAULT);
    if (ret)
        return ret;
...

I modified device tree configuration to use compile-time constants as attached in one of previous posts:

static int dp83869_hardcoded_init(struct phy_device *phydev) {
	struct dp83869_private *dp83869 = phydev->priv;

	printk(KERN_INFO "%s\n", __FUNCTION__);

	dp83869->io_impedance   = -EINVAL;
	dp83869->clk_output_sel = DP83869_CLK_O_SEL_REF_CLK;
	dp83869->mode           = DP83869_RGMII_1000_BASE;
	dp83869->io_impedance   = 0x10; //50 ohm
	dp83869->port_mirroring = DP83869_PORT_MIRRORING_DIS;
	dp83869->rx_fifo_depth  = DP83869_PHYCR_FIFO_DEPTH_4_B_NIB;
	dp83869->tx_fifo_depth  = DP83869_PHYCR_FIFO_DEPTH_4_B_NIB;

	return 0;
}

---

Aniruddha Khadye said:

Are you using boot strap resistors to directly enter into Fiber mode at start-up or do you start-up in a different mode and activate Fiber mode through registers.

We use both strap resistors and register configuration. Both methods set the same mode: DP83869_RGMII_1000_BASE .

---

Aniruddha Khadye said:

If you are activating fiber mode through register, can you try adding a soft restart instruction at the end of your Fiber mode initialization script. Soft restart can be done by writing 0x4000 to register 0x1F. Soft restart can be done by writing 0x4000 to register 0x1F.

Sure, I've added following code to the end of "dp83869_config_init":

static int dp83869_config_init(struct phy_device *phydev)
{
...
    /* perform soft restart                       reg, value */
    ret = phy_write_mmd(phydev, DP83869_DEVADDR, 0x1F, 0x4000);
    if(ret)
        return ret;

    printk(KERN_INFO "%s - soft restart done, 0x%x written to 0x%x\n", __FUNCTION__, 0x4000, 0x1F);

    return ret;
}

No success yet.

---

Aniruddha Khadye said:

Regarding your Q1, we have tested our own boards with linux at the time of the linux driver development.

Great, can you please share your hardware and software setup, strap resistor configuration, PHY register dump, device tree with phy node and PHY driver source?

---

Aniruddha Khadye said:

For Q2, BMSR register bit 2 is Latch Low bit. So when it transitions from low to high it needs to be read twice.

• I modified MAC driver to allow PHY register dump via sysfs file read and dmesg. Is my assumption correct that 3 register dumps in a row should read "link up bit" as 1 if the PHY recognizes LINK UP?
  
  

  
  [72935.121913] lan78xx_mdiobus_read:  *(0x0004, ANAR                  ) = 0x0580
  [72935.122644] lan78xx_mdiobus_read:  *(0x0001, BMSR                  ) = 0x6169 //here
  [72935.123344] lan78xx_mdiobus_read:  *(0x0000, BMCR                  ) = 0x1140
  [72935.124039] lan78xx_mdiobus_read:  *(0x0001, BMSR                  ) = 0x6169 //here
  [72935.124658] lan78xx_mdiobus_read:  *(0x0009, GEN_CFG1              ) = 0x1200
  [72935.125274] lan78xx_mdiobus_read:  *(0x0000, BMCR                  ) = 0x1140
  [72935.125887] lan78xx_mdiobus_read:  *(0x0001, BMSR                  ) = 0x6169 //here
  [72935.126476] lan78xx_mdiobus_read:  *(0x000A, GEN_STATUS1           ) = 0x0000
  [72935.127069] lan78xx_mdiobus_read:  *(0x0000, BMCR                  ) = 0x1140
  [72935.127664] lan78xx_mdiobus_read:  *(0x0001, BMSR                  ) = 0x6169 //here
  [72935.128267] lan78xx_mdiobus_read:  *(0x0005, ALNPAR                ) = 0x4060
  [72935.128951] lan78xx_mdiobus_read:  *(0x0001, BMSR                  ) = 0x6169 //here
  [72935.129552] lan78xx_mdiobus_read:  *(0x000A, GEN_STATUS1           ) = 0x0000
  [72935.130179] lan78xx_mdiobus_read:  *(0x0005, ALNPAR                ) = 0x4060

• In attached MDIO communication dump I see Linux (probably generic-phy kernel layer) reading the BMSR multiple times. If "latch low" LINK UP bit is standard PHY behaviour, the generic-phy layer should take care of this since many other network cards work with the generic-phy layer correctly.

---

Aniruddha Khadye said:

Is it possible to change your link recognition algorithm to read the register twice?

It's not possible since generic-phy layer is being used for this purpose (at least i think it's generic phy). If there is any non-standard behaviour or any extra setup needed, then custom phy driver is required to handle that. I suppose this is the reason why TI's PHY Linux driver for dp83869hm exists. I don't understand why there is no additional setup in the TI Linux driver when PHY operates in optical modes:

static int dp83869_configure_mode(struct phy_device *phydev,
                  struct dp83869_private *dp83869)
{
...
    switch (dp83869->mode) {
    case DP83869_RGMII_COPPER_ETHERNET:
        ret = phy_write(phydev, MII_DP83869_PHYCTRL,
                phy_ctrl_val);
        if (ret)
            return ret;

        ret = phy_write(phydev, MII_CTRL1000, DP83869_CFG1_DEFAULT);
        if (ret)
            return ret;

        ret = dp83869_configure_rgmii(phydev, dp83869);
        if (ret)
            return ret;
        break;
    case DP83869_RGMII_SGMII_BRIDGE:
        ret = phy_modify_mmd(phydev, DP83869_DEVADDR, DP83869_OP_MODE,
                     DP83869_SGMII_RGMII_BRIDGE,
                     DP83869_SGMII_RGMII_BRIDGE);
        if (ret)
            return ret;

        ret = phy_write_mmd(phydev, DP83869_DEVADDR,
                    DP83869_FX_CTRL, DP83869_FX_CTRL_DEFAULT);
        if (ret)
            return ret;

        break;
    case DP83869_1000M_MEDIA_CONVERT:
        ret = phy_write(phydev, MII_DP83869_PHYCTRL,
                phy_ctrl_val);
        if (ret)
            return ret;

        ret = phy_write_mmd(phydev, DP83869_DEVADDR,
                    DP83869_FX_CTRL, DP83869_FX_CTRL_DEFAULT);
        if (ret)
            return ret;
        break;
    case DP83869_100M_MEDIA_CONVERT:
        ret = phy_write(phydev, MII_DP83869_PHYCTRL,
                phy_ctrl_val);
        if (ret)
            return ret;
        break;
    case DP83869_SGMII_COPPER_ETHERNET:
        ret = phy_write(phydev, MII_DP83869_PHYCTRL,
                phy_ctrl_val);
        if (ret)
            return ret;

        ret = phy_write(phydev, MII_CTRL1000, DP83869_CFG1_DEFAULT);
        if (ret)
            return ret;

        ret = phy_write_mmd(phydev, DP83869_DEVADDR,
                    DP83869_FX_CTRL, DP83869_FX_CTRL_DEFAULT);
        if (ret)
            return ret;
        break;
    case DP83869_RGMII_1000_BASE:
    case DP83869_RGMII_100_BASE:
/* HERE - EMPTY */
        break;
...
}

This means no setup for RGMII impedance etc is made.

---

Aniruddha Khadye said:

This might also explain why when you read the registers it shows Link up but Linux target report link down

Unfortunatelly this seems unlikely to me, the only time I got Link UP was when I manually restarted auto negotiation (via sysfs file), but, router "Optical interface LED" usually turns off at this point and PHY reports "LINK UP". All other cases (no manual auto negotiation restart) always end up with PHY reporting "LINK DOWN" in BMSR, bit 2.

Without additional auto negotiation restart, the link partner reports "Link ok" and "running", rx power "-5.953 dBm",

Auto Negotiation

done

Rate

1Gbps

Full Duplex

yes

Tx/Rx Bytes 49.9 MiB / 0 B

RX bytes is 0. TX bytes increases at rate approximatelly 242 bps.

---

I found following differences among PHY datasheet and my observations how the PHY acutally works:

• PHY datasheet, 9.3.6 Interrupt: "Both the interrupt status registers must be read in order to clear pending interrupts. Until the pending interrupts are cleared, new interrupts may not be routed to the interrupt pin."

• PHY datasheet, 9.4.2.1 1000BASE-X: "In fiber mode, the speed is not decided through auto-negotiation. Both sides of the link must be configured to the same operating speed."

Questions:

1. If auto negotiation is not used to select link speed, what is it used for?
2. Should it be enabled or disabled for RGMII-to-1000BASE-X ?
3. Do I understand correctly that the PHY operating in RGMII-to-1000BASE-X mode has been tested and verified under Linux?
4. Who should provide working PHY Linux driver for operation in RGMII-to-1000BASE-X mode?
5. In the TI's PHY Linux driver, I see only "INTERRUPT_STATUS" register is being read and "FX_INT_STS" register is untouched. Is this correct?
6. Can you please share hardware and software setup, strap resistor configuration, PHY register dump, device tree with phy node and PHY driver source from the time the TI developed PHY driver for DP83869 (or any up to date version)?

Best regards,

Michal.

Hi Aniruddha,

is there anything new on this topic? Is there a chance I'll get some reply that will help solve the problem within one week from now?

Best regards,

Michal.

Hi Aniruddha,

if there is anything new on this topic please let me know, I'm stuck here.

Best regards,

Michal.

Hi Michal,

To answer your questions, autonegotiation process exchanges other information like half/full duplex, remote fault etc. DP83869HM doesn't adjust speed through auto-negotiation process when in fiber mode. As an experiment you can try and disable fiber autonegotiation to force it in to 1Gbps speed.

Here is an E2E thread for the linux code, header file and device tree documentation: 

We used straps to verify to configure the PHY in Fiber mode when the driver was being developed. It would be difficult to get the register dump from the time of development. However there is an example on E2E where DP83869 fiber was implemented with AM5728. Here is the E2E thread where they got the Fiber  application working by using the linux driver, disabling the autonegotiation, and changing the delay on the TX side: 

Is it possible for you to share the schematic of your application? If putting them on the web is not an option, you can send it over E2E private chat to me.

-Regards

Aniruddha

Hi Aniruddha,

thanks for links, I've carefully reviewed what you've sent.

I see there is newer dp83969 Linux driver with tx and rx delays, strap configuration support and more nice features.

Unfortunately symptoms are the same:

* auto-negotiation completed

* link down

I confirm that BMSR link status is read twice, generic phy drivers are aware of link status being latched low.

I attach the schematic via private chat.

Best regards,

Michal.

Hi Aniruddha,

we decided to attach the shematic there.

/cfs-file/__key/communityserver-discussions-components-files/138/PCBA_2D00_02_2D00_03_2D00_02V01_2D00_forTI.pdf

Best regards,

Michal.

Hi Michal,

Thanks for sending the schematics, I have also accepted your request so we can share more details through private chat if required. Allow me a day to review the schematics and I will let you know my feedback.

-Regards

Aniruddha

Hi Michal,

I took a look at the schematic and noticed something that I wanted to clarify. I saw that the VDDIO is set to 1.8V via resistor R22 and the fiber optic is at 3.3V. The signal detect (LOS) pin from the fiber transceiver is electrically connected to the PHY and it is also pulled up to VDDIO. For debug experiment, is it possible to try with VDDIO = 3.3V via R21. Is the pull up on Signal Detect pin required from the Fiber Transceiver datasheet? If not, you can remove that. Please verify that the signal detect polarity of the PHY is configured according to the output polarity of the LOS pin of the Fiber Transceiver.

-Regards

Aniruddha

Hi Aniruddha,

we are testing your suggestion.

Regards,

Michal.

Hi Aniruddha,

we have been testing your suggestion on different hardware revisions and we came to these consistent results:

1) The network card works as expected when I use modified kernel that changes link down status to link up in software (reports fake link up). The network card works with the modified kernel in all SD pullup and VDDIO configurations mentioned before. The same "fake link up" test has been already done in the past but with buggy hardware, so with no success.

2) The phy reports link down and works with fake link-up kernel:

 * when both SD pullup is present or absent,

 * when VDDIO is set to 1V8 or 3V3.

Aniruddha Khadye said:

Please verify that the signal detect polarity of the PHY is configured according to the output polarity of the LOS pin of the Fiber Transceiver.

I confirm the LOS pin of the Fiber Transceiver is active low. Although the PHY datasheet does not literally mention the SD pin is active low, it seems it is - the SD pin is WPU (weak pull up) type.

Since the link status detection ( https://www.ti.com/lit/an/snla305/snla305.pdf?ts=1602503485675 ) seems to be the last unresolved issue, do you have any idea how to implement it to the linux kernel or the PHY driver?

Best regards,
Michal.

Hi Aniruddha,

I would like to follow up. Is there anything new?

The optical network card works for one week now with "FAKE LINK UP" kernel.

Best regards,

Michal.

Hi Michal,

I had some further questions about the LOS signal of the Fiber Transceiver, is my understanding below correct?

Fiber Cable disconnected, LOS = LOW

FIber cable connected, LOS = HIGH

As you correctly said, SD pin of the PHY is expects a active LOW input so I just wanted to make sure that the LOS signal meets the SD requirement. To clarify,

PHY expects SD = LOW when Fiber cable is connected

PHY expects SD = HIGH when Fiber cable is disconnected

The suggestion in the app note is to implement a polling method to read the sync and fiber link status bit. Unfortunately, actual software implementation would be out of scope for this forum. 

-Regards

Aniruddha