DP83867IR: ethernet link instability in SGMII mode

Hi Jens,

We will discuss this internally with the team and provide you an response later this week.

--

Regards,

Hillman Lin

Hi Jens,

From your second registers dump, I can see that your link partner only does not support 100mbps and 1000mbps through register 0x000A bit[11:10] and register 0x0005 [8:5].

It seems like it is the link partner issue that is not supporting the speed. Could you check register 0x0004 and 0x0009 on link partner to make sure they support or advertise the required speed?

--

Thank you,

Hillman Lin

Hi Hillman Lin,

thanks for your support!

You are right, in second register dump only 10Mbps is supported by the link partner. The link partner is prepared for 10/100/1000Mbps but I have limited the speed by the ethtool command: ethtool -s enp4s0 speed 10 duplex full

Thats the reason why the link partner shows only 10Mbps support.

1. Register dump observed with ethtool -s enp4s0 speed 1000 duplex full => ping working

2. Register dump observed withethtool -s enp4s0 speed 10 duplex full => ping failure

If the ping fails, the Reg8 =0, which means there is no link code word received? But Reg37=0x3 shows Autonegotiation completed and SGMII page has been received.

Br,

Jens

Hi Hillman Lin,

When I reduce the speed to 10Mbps by using the ethtool command on the link side, negotiation is restarted. Therefore I would like to know why it is not working.

Br,

Jens

I would like point to the enabled error bits. Maybe this can give a hint for further analysis.

Reg 0x13 Interrup status: Bit2 XGMII Error Bit =1

Reg 0xA Status : Bit 13= 1 //local receiver not OK

                            Bit 12 =1 //remote receiver not OK

Any idea would could be the root cause?

Br,

Jens

Hi Jens,

• 0x0013 means there is some error on the MAC interface that is triggering the interrupt.
• 0x000A means your system is not auto negotiation complete because you are not able to detect remote receiver.

May I ask is the PHY issue resolve? Why are you receiving this register values? 0x000A should not trigger when link is up.

--

Regards,

Hillman Lin

Hi Hillman Lin,

The PHY issue isn't resolved. The link is up but in most of the cases the ping goes fail. The behaviour is strange because the link is up and the error bits in the status register are enabled. We have measured the pulses for autonegotiation and it looks good.

How can we go further?

Br,

Jens

Hi jens,

Just want to confirm, could you read register 0x0001 multiple times when Local receiver and remote receiver is not ok? We are expecting the link is not up when the receiver is not ok. This could give us indication on if the issues is on MDI interface or SGMII interface. 

If possible, could you also let me know which register did you read that indicate local receiver and remote receiver is not ok?

--

Regards,

Hillman Lin

Hi Hillman Lin,

I read the registers multiple times and the content didn't change.

Reg 0x1 = 0x796d

=> valid link, no remote faul detected, autonegotiation completed

Reg 0xa = 0x0

=>Local/Remote receiver not ok

Reg 0x37=0

SGMII page not received, AN not completed

Would that mean the MDI interface is ok and the problem is on the SGMII IF? 

Br,

Jens

Hi Jens,

Yes, that means the issue is in SGMII side. One more thing I would like to confirm. DP83867IR does not support SGMII communication, are you using DP83867IS or DP83867 IR?

There are two things that we can would like to check for SGMII communication if you are using DP83867IS:

• Could you check register 0x0010?
• Are you able to probe both SOP, SON, SIP, and SIN on SGMII? If possible, could you also let me know the position where you probe the SGMII?
• Could you also check what are the length match between all the traces in SGMII communication?

--

Regards,

Hillman Lin

Hi Hillman Lin,

The correct label of the used chip:DP83867ISRGZT, where  SGMII should be supported.;-)

Content of Reg0x10:0x5848

We will measure all proposed signals and will provide the result tommorow.

Would it makes sense to check the data path with the BIST mode? I could enable the MI reverse loopback and the BIST packet generator would verify the data integrety. If there is a problem with the SGMII path, the test should be successfully. What do you think?

Br,

Jens

Hi Jens,

Got it, we will wait for your probing result tomorrow. The main thing we are looking for are the peak to peak voltage on the SGMII communication.

BIST packet generator path mainly goes to MDI side instead of SGMII side. One thing that you could do to confirm the issue is on SGMII side is enable packet generator on MAC or Soc. Then enable MII loopback on DP83867 and see if the SoC is able to receive back the packets.

--

Regards,

Hillman Lin

Hi Hillman Lin,

I tried to enable the BIST packet generator, not sure if my understanding is correct. Here my setup for the "good case":

I have taken DP83867 as link partner, which is configured for RGMII and reverse loopback enabled

1.HW Reset: 0x1F 0x8000

2.AN disabled: 0x0 0x140

3.enable packet generator: 0x16 0x20

4. enable loopback bit (Bit14)in register 0x0

DP83867 configured for RGMII and packet generation enabled:

1.HW Reset: 0x1F 0x8000

2.AN disabled: 0x0 0x140

3. loopback configuration: 0xFE 0xE720

4.enable packet generator: 0x16 0x5000

5. enable loopback bit (Bit14)in register 0x0

Result:

Reg17:0x240  =>packet generation in progress, PRBS checker not locked

Reg72: 0x0

Could you please review used sequence?

Attached the screenshoots of  „SOP“, „SON“

Screenshot 1:

On a PHY EVM (Evaluation Board from Texas Instruments) we measured SOP/SON at the PHY output.

Screenshot 2:

In our application we removed the MAC. So the PCB traces from PHY to MAC are accessible. On this MAC’s input we measured SOP/SON.

Both waveforms look very similar, so our interpretation is: The RX signal path on our PCB is fine.

Br,

Jens

Hi Jens,

Thank you for sharing the waveform. It seems like the SGMII signal looks good to me on SOP/SON.

• Have you also checked SIP/SIN and see if you are able to receive any signal?

One more thing I would like to double check. Did you power the processor and PHY at the same time?

• If possible, could you do a software reset 0x001F to 4000 or hardware reset 0x001f to 8000 and see if that help with the SGMII link up?

--

Thank you,

Hillman Lin

Hi Hillman Lin,

Currently we measure SIP/SIN signals and will update you if we are ready.

I would like to share a picture of my current test setup to verify the PHY-MDI path by using the BIST packet generator. Could you please check if the configuration is correct? 

test_setup.pptx

Br,

Jens

Hi Jens,

We will wait for your response on SIP/SIN measurement.

The procedure seems correct to me. Do make sure to enable reverse loopback first. May I ask what is the purpose for this test? 

If possible, could you try the following test? This could confirm whether issue are on SGMII side.

One more thing I would like to double check. Did you power the processor and PHY at the same time?

• If possible, could you do a software reset 0x001F to 4000 or hardware reset 0x001f to 8000 and see if that help with the SGMII link up?

--

Regards,

Hillman Lin

Hi Hillman Lin,

Thanks for reviewing the BIST configuration.
We tried already to add a HW/SW reset but we observed no difference.
Yes, reverse loopback has been enabled before the packet generator is sending packets.
The purpose of the test was to verify the data path PHY-MDI without involving SGMII.
Here my understanding how my test setup should work.
The packet generator is transmitting packets to the link partner, which is looping back the data caused by the reverse loop configuration. SGMII shouldn’t be involved in this case. In my test, the BIT Register0x17[11] is low, which means the data paths PHY – MDI is not valid.
Please correct me if I’m wrong.
I have checked your proposal and it seems identically to my test setup. Did you forgot to add something or do I miss the crucial thing? Maybe you can clarify it.
BTW: What is the meaning of BIT14 in Register 0x0?

Br,

Jens

Attached the results of the  SIP/SIN measurement and the result of Intra-differential-pair trace differences:

SOP/SON   0,88mm

SIP/SIN   0,08mm

Please advice how to go further.

Br,

Jens

Hi Jens,

Sorry if my explanation is not clear on BIST test setup. If possible, could you first enable the reverse loopback on the DP83867 that connected with SGMII interface. Then enable the BIST generator on the DP83867 that has the RGMII interface? This could bypass the SGMII interface.

From your waveform, it seems like the SGMII input to the PHY is not really clean. Could you also double check on the layout checklist xMII session to see if you are meeting the recommendation?

4774.IndustrialPHY_Layout Review Checklist.xlsx

--

Thank you,

Hillman Lin

Hi Hillman Lin,

I got the same result with enabled BIST generator on the DP83867 with RGMII interface. The  BIT Register0x17[11]  isn't enabled.

Attached the layout checklist.4774.IndustrialPHY_Layout Review Checklist_kontron.xlsx

Br,

Jens

Hi Jens,

Register 0x0017 bit[11] should be lock if you enable BIST generator. If possible, could you see if can enable analog loopback through register 0x0016 and then enable BIST generator and see if you are able to run the BIST successfully? 

I take a look at the layout checklist. It seems like the layout should be fine on SGMII interface. I also put some comment on the Rbias pin trace, but this shouldn't have much effect on SGMII interface.

5873.IndustrialPHY_Layout Review Checklist_kontron.xlsx

--

Regards,

Hillman Lin

Hi Hillman Lin,

yes, with enabled analog loopback the BIST runs successfully. 

What does it mean for further analysis? 

There is a reverence board (DP83867ERGZ-S-EVM) available, which I could also use to run the BIST generator.

Thanks for layout review!

Br,

Jens

Hi Jens,

Currently, Someone in our team is looking at the driver for SGMII interface. Based on all the experiment we did previously, we do think the issue occur on SGMII side. We can continue the conversation on SGMII driver make sure the driver is working properly. An software expert in our team will continue helping you SGMII driver for DP83867.

--

Thank you,

Hillman Lin

Hi Hillman Lin,

we are confused, because the BIST generator test with the reverse loop on the link partner side wasn't successfully. This test validates the PHY-MDI data path and excludes the SGMII IF. Maybe you can give us more details.

Could you please clarify the meaning of Reg17[Bit7] and Reg17[Bit6]. It would help to understand the functionality of the BIST mode better.

Br,

Jens

Hi Jens, 

Thank you for trying out. 

One more thing that we could try on your side is enable the packet generate on the MAC side like the following:

MAC  < -- RGMII -- > 867EVM1 < --- Copper --> 867EVM2 <-- reverse loopback --> 867EVM1 < -- RGMII -- > MAC

This could also give you an indication if the copper is working properly.

From my understanding, PRBS should work. However, I don't think that is where the main issue occur since SGMII auto-negotiation is not complete. If there is an faster way to confirm. The method above should give you the similar result.

Reg0x0017 bit[7] and bit [6] should not have an effect on BIST test.

--
Regards,

Hillman Lin

Hi Hillman Lin,

I have some good news! All problems are gone with disabled "Energy Efficiency Ethernet" mode on the link partner side.

#ethtool --set-eee enp3s0 eee off

The register settings looks fine as well, there are no indications for errors.

Do you have some experience with this EEE mode issue already?

Br,

Jens

Hi Jens,

Thank you for sharing the good news. We never seen this issue before, I will discuss with the team to have their input on this. 

• May I ask when you said link partner, are you talking about one of the DP83867PHY is configure in EEE mode?
• The issue lies on the copper side instead of SGMII side?

Just want to have a better picture before I propose the observation to the team.

--

Thank you,

Hillman Lin