AM5748: AM5748: Issue with PHY DP83822 driver in TI SDK 6.02

Here is the DTS config for PRUSS1 and 2:

Bold text is new addition in SDK 6.02 BSP (using am574x-idk.dts as an example):

Hi Paritosh,

I will take a look at this and get back with you.  Do you happen to have a TI AM574 IDK available to test if this issue is reproducible with the vanilla SDK?

Regards,
Mike

Hi Michael,

Thanks for your reply.

I have IDKs but it may not be possible to reproduce issue 1 on the IDK as this issue is specific to PHY DP83822. As per dmesg on IDK, it seems to have KSZ9031 and TLK10X PHYs.

Paritosh,

Ah, thank you.  I'll ping the two driver authors and see if they can help resolve this difference in behavior and what additional considerations might exist.

Regards,
Mike

Hi Paritosh,

Thank you for your patience.

We created a simple script for the open source program "mdio-tool" (github link: https://github.com/PieVo/mdio-tool).

Please find the attached script and pre-compiled mdio-tool binary.  Feel free to clone the git repo and rebuild on the target yourself (no fancy compiler options required)

Once you're ready, run the script in the same path as the mdio-tool on your target, and attach the log files for us to analyze.  The script takes a single parameter of the network interface name, e.g.: "./mdioregdump-dp83822.sh eth2".

It would be good to see the register dumps prior to link-up, successful cable detect and link-up, cable disconnected after link-up, and cable connected with failed cable detect.

Regards,
Mikehttps://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/4118.mdio_2D00_toolhttps://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/2451.mdiodump_2D00_dp83822.sh

Paritosh,

FYI, we aren't entirely sure exactly why you're not getting interrupts at the moment.  This testing is to capture the state of the PHY at various times.  If you can do that for the working and non working versions then hopefully we can identify whether the difference relates to the PHY itself.  If we cannot find any significant differences in the PHY configuration, then we'll need to turn our attention to the processor instead.

Along those lines, have you done "cat /proc/interrupts" for the different scenarios?  Is there any obvious difference there?

Brad

Hi Brad and Mike,

Logs are attached in 3 folders:

- working_dp83848_c_driver : Working case when using dp83848.c driver

- not-working_dp83822_c_driver : Case where issue is reporduced with dp83822.c driver

- working_dp83822_c_driver : working case with dp83822.c driver (See note below).

/proc/interrupts output is included with log files.

1667.LOGS.zip

Note:

Observed that if the mdiodump-dp83822.sh tool is run when cable is disconnected, this fixes the issue. That is, device detects cable connect and dis-connect. Also observed that this fix works only for couple of cable connect / dis-connect cycles, and after that device again fails to detect cable connect and dis-connect.

Due to this, in folder not-working_dp83822_c_driver, there is only single log file (after the case when issue is reproduced). Because, issue can not be reproduced if tool is run.

Please let me know if this is confusing. I will try to provide more clarity.

Thanks,
Paritosh

Paritosh,

Thank you for providing the logs and your observation that running the MDIO tool seems to "fix" the issue.  I think this is a very valuable clue.

We will get back with you by the end of the week.

Regards,
Mike

Hi Brad,

Setting it to IRQ_TYPE_LEVEL_LOW is fixing the issue. Cable disconnect/re-connect are detected.

When cable is not connected this interrupt count keeps going up; faster for some PHYs that others; 

root@am57xx-evm:~# cat /proc/interrupts | grep mdio
191: 2815 0 48053000.gpio 0 Level 4b232400.mdio:01
192: 1863 0 48059000.gpio 3 Level 4b232400.mdio:03
197: 1893 0 48059000.gpio 2 Level 4b2b2400.mdio:05
198: 3728 0 48057000.gpio 30 Level 4b2b2400.mdio:1f

For EDGE case as well, they keep going up; but not for the PHY, that had link up earlier.

root@am57xx-evm:~# cat /proc/interrupts | grep mdio 
191: 44 0 48053000.gpio 0 Edge 4b232400.mdio:01
192: 5815 0 48059000.gpio 3 Edge 4b232400.mdio:03
197: 5789 0 48059000.gpio 2 Edge 4b2b2400.mdio:05
198: 5805 0 48057000.gpio 30 Edge 4b2b2400.mdio:1f

Thanks,

Paritosh

Paritosh,

Paritosh Dixit said:

For EDGE case as well, they keep going up; but not for the PHY, that had link up earlier.

I think you're saying that the interrupts stop incrementing for the PHY which is unable to reconnect.  Did I understand correctly?  That makes sense with my understanding of the issue.  From a hardware standpoint, I expect you will observe the the interrupt line is permanently asserted low in the case where you no longer can connect.  This is because the PHY is trying to trigger an interrupt, but the processor doesn't see it because there's no edge.

The reason you didn't see the interrupt originally is because fewer interrupts were enabled and so there was less chance of hitting the timing corner case for this issue to occur.

I think the issue is resolved now, right?  Is there any other related question that we missed?

Best regards,
Brad

Hi Brad,

I did some more tests today for interrupt count in SDK 5.02 (driver dp83848.c) and SDK 6.02 (drivers dp83848.c and dp83822.c) for EDGE and LEVEL_LOW triggers.

Here is the summary:

Logs:

Paritosh,

First, based on your earlier register dumps that you shared, I can see a major difference with respect to which interrupts are being enabled in these scenarios.  There are lots more interrupts being enabled with the new driver.  However, it looks like they're not being fully handled within the driver. In particular, the function dp83822_ack_interrupt() is just doing a very rudimentary reading and clearing of the bits in MISR1 and MISR2.  It's not taking any action beyond that.  If you want to really dig into what's happening, I suggest adding a printk in that function so you can see which interrupts are pending.  My suspicion is that some of those interrupts might require deeper handling than reading MISR1/MISR2 in order to clear them.  It would probably be useful to understand which ones are causing you issues.

Another option (or another test) would be to adjust the interrupt enables for the DP83822 driver to match your original interrupt configuration.  I haven't tested this, but inspecting the bits of your register dumps, I believe this is the appropriate change to get the same interrupt configuration:

diff --git a/drivers/net/phy/dp83822.c b/drivers/net/phy/dp83822.c
index 6e8a2a4f3a..9a7da569bf 100644
--- a/drivers/net/phy/dp83822.c
+++ b/drivers/net/phy/dp83822.c
@@ -203,36 +203,15 @@ static int dp83822_config_intr(struct phy_device *phydev)
if (misr_status < 0)
return misr_status;

- misr_status |= (DP83822_RX_ERR_HF_INT_EN |
- DP83822_FALSE_CARRIER_HF_INT_EN |
- DP83822_ANEG_COMPLETE_INT_EN |
+ misr_status |= (DP83822_ANEG_COMPLETE_INT_EN |
DP83822_DUP_MODE_CHANGE_INT_EN |
DP83822_SPEED_CHANGED_INT_EN |
- DP83822_LINK_STAT_INT_EN |
- DP83822_ENERGY_DET_INT_EN |
- DP83822_LINK_QUAL_INT_EN);
+ DP83822_LINK_STAT_INT_EN);

err = phy_write(phydev, MII_DP83822_MISR1, misr_status);
if (err < 0)
return err;

- misr_status = phy_read(phydev, MII_DP83822_MISR2);
- if (misr_status < 0)
- return misr_status;
-
- misr_status |= (DP83822_JABBER_DET_INT_EN |
- DP83822_WOL_PKT_INT_EN |
- DP83822_SLEEP_MODE_INT_EN |
- DP83822_MDI_XOVER_INT_EN |
- DP83822_LB_FIFO_INT_EN |
- DP83822_PAGE_RX_INT_EN |
- DP83822_ANEG_ERR_INT_EN |
- DP83822_EEE_ERROR_CHANGE_INT_EN);
-
- err = phy_write(phydev, MII_DP83822_MISR2, misr_status);
- if (err < 0)
- return err;
-
physcr_status = phy_read(phydev, MII_DP83822_PHYSCR);
if (physcr_status < 0)
return physcr_status;

Best regards,
Brad

Hi Brad,

With this patch, the behavior of DP83822 driver is same as DP83848 driver, as far as MDIO interrupts are concerned. There are no continuous interrupts when cable is removed; and interrupts are observed only when cable is connected/disconnected.

Also, DP83822 driver is able to detect the cable connects/disconnects with EDGE and LEVEL_LOW both triggers. This is in-line with your analysis that additional interrupts were causing the interrupt timing issue.

With patch, prints of MISR1 and MISR2 from dp83822_ack_interrupt() for PHY1:

Cable disconnect
[ 273.505788] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x6224
[ 273.512323] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0x0
[ 273.524419] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x24
[ 273.530770] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0x0

Cable connect:
[ 279.697394] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x6424
[ 279.703934] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0x2800
[ 279.719087] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x24
[ 279.725447] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0x0

Without patch (with additional interrupt enables), prints of MISR1 and MISR2 from dp83822_ack_interrupt() from PHY1:

[ 44.648146] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x42ff
[ 44.654682] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0xff
[ 44.661866] prueth pruss1_eth eth0: Link is Down
[ 44.666808] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x22ff
[ 44.673334] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0xff
[ 44.680961] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0xff
[ 44.687312] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0xff
[ 44.834100] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0xff
[ 44.840453] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0x8ff
...
...
[ 48.375376] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0xff
[ 48.381728] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0x20ff
[ 48.389351] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0xff
[ 48.395729] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0xff
[ 48.403882] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x40ff
[ 48.410411] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0xff
[ 48.418133] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0xff
[ 48.424486] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0xff
[ 48.485263] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x40ff
[ 48.491786] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0xff
[ 48.498981] prueth pruss1_eth eth0: Link is Up - 100Mbps/Full - flow control off
[ 48.506731] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x24ff
[ 48.513266] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0xff
[ 48.520939] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0xff
[ 48.527288] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0xff

Does this indicate a issue in device HW or configuration?

Thanks,
Paritosh

Can you post your code for the dp83822_ack_interrupt() function?

static int dp83822_ack_interrupt(struct phy_device *phydev)
{
        int err;

        err = phy_read(phydev, MII_DP83822_MISR1);
        if (err < 0)
                return err;

        if ( phydev->mdio.addr == 1 )
                dev_err(&phydev->mdio.dev, "MII_DP83822_MISR1 : 0x%x\n", err);

        err = phy_read(phydev, MII_DP83822_MISR2);
        if (err < 0)
                return err;

        if ( phydev->mdio.addr == 1 )
                dev_err(&phydev->mdio.dev, "MII_DP83822_MISR2 : 0x%x\n", err);

        return 0;
}

Paritosh Dixit said:

With patch, prints of MISR1 and MISR2 from dp83822_ack_interrupt() for PHY1:

Cable disconnect
[ 273.505788] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x6224
[ 273.512323] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0x0
[ 273.524419] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x24
[ 273.530770] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0x0

The only interrupts that are actually enabled above are:

• MISR1 Bit 2: Autonegotiation complete enable
• MISR1 Bit 5: Link Status Change enable

The following are pending:

• MISR1 Bit 9: False Carrier Counter Half-Full Interrupt
• MISR1 Bit 13: Link Status changed
• MISR1 Bit 14: Energy Detect interrupt

Paritosh Dixit said:

Without patch (with additional interrupt enables), prints of MISR1 and MISR2 from dp83822_ack_interrupt() from PHY1:

[ 44.648146] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR1 : 0x42ff

All interrupts are enabled, and the following are pending:

• MISR1 Bit 9: False Carrier Counter Half-Full Interrupt
• MISR1 Bit 14: Energy Detect interrupt

We might need commentary from the PHY team on the significance of the False Carrier interrupts, e.g. is something wrong with the hardware that we're getting those interrupts?

I suspect in order to make it go away (fully) we likely need to read register 0x0014 False Carrier Sense Counter Register (FCSCR).  That would reset the counter which I expect will clear the error condition.  Paritosh, could you try that as an experiment, i.e. in the interrupt handler if you see MISR1 Bit 9 being set then can you in turn read from register 0x0014?  That should reduce the number of interrupts if that's correct.  Or alternatively perhaps only disable the False Carrier Counter Half-Full Interrupt.  Even though you're seeing the energy detect interrupt, I don't think there's any other action specifically needed to clear that.

Hi Brad,

I tried to read FCSCR based on False Carrier Sense interrupt (MISR1 Bit 9). It does clear this interrupt. Still, MISR2 Bit 11 (MDI Crossover Change Interrupt) keeps coming.

[  201.255675] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0x8ff

Tried another test: By just disabling the "MDI Crossover Change Interrupt Enable" in MISR2 (P83822_MDI_XOVER_INT_EN), these continuous interrupts are stopped.

What could be right mechanism to clear this interrupt; I tried to find in DP83822 datasheet, but could not make much sense of it.

Thanks,
Paritosh

Paritosh Dixit said:

I tried to read FCSCR based on False Carrier Sense interrupt (MISR1 Bit 9). It does clear this interrupt.

Ok, great!  Perhaps the best solution in this case is to keep that particular interrupt enabled, and add a few lines to handle it, e.g. read FCSCR and perhaps log an error (such as outputting that you received a False Carrier Sense Interrupt, and perhaps output the value of FCSCR and/or MISR1).  If you do that, are you just seeing this as a one time error at start-up or do you see it consistently over a period of time?  I'm still waiting to hear back as to the underlying meaning of this error, so for now I think simply being able to log it and clear it would be a good start.

Paritosh Dixit said:

till, MISR2 Bit 11 (MDI Crossover Change Interrupt) keeps coming.

[  201.255675] TI DP83822 4b232400.mdio:01: MII_DP83822_MISR2 : 0x8ff

Tried another test: By just disabling the "MDI Crossover Change Interrupt Enable" in MISR2 (P83822_MDI_XOVER_INT_EN), these continuous interrupts are stopped.

What could be right mechanism to clear this interrupt; I tried to find in DP83822 datasheet, but could not make much sense of it.

I was expecting that simply reading MISR2 would clear the interrupt.  We will need clarification from the PHY team.  I recommend just disabling that one altogether for now.  

Hi Brad,

False Carrier Sense interrupt is received once every time cable is disconnected. Added below log:

[   56.436701] TI DP83822 4b232400.mdio:01: Interrupt FALSE_CARRIER_HF Received. MISR1 : 0x62ff, FCSCR : 0x0

Disabled "MDI Crossover Change Interrupt Enable" in MISR2 for now.

Thanks,
Paritosh

Paritosh,

I started a thread in the Interface forum to get some specific details on the DP83822 behavior:

https://e2e.ti.com/support/interface/f/138/t/893985

I recommend that we move this discussion there as appropriate.  At this point from the processor perspective I think we understand which interrupts are causing the issues.  The only remaining actions are to determine whether to disable some of these interrupts entirely or put in some additional handling.  We can always post a summary here once we have come to some conclusions in the other thread.

Best regards,
Brad