DP83867ERGZ-S-EVM: EVM unable to make SGMII connection work

Hi Phil,

Thank you for the query and the detailed analysis.

The LED_0 value of 1.1V wouldn't have damaged the pin/device.

It would be difficult to debug without register access. But, let's try a few things before you receive MSP430.

Can you please check signals on the following pins?

1. Voltage on unconnected RX_CLK pin with DMM after link-up
2. Signals on SGMII_SOP, SON and their differential voltage observed on an oscilloscope
3. Voltages of LED_0, LED_1, LED_2 after link-up

--
Regards,
Gokul.

Gokul,

Thanks for the response.

1.  RX_CLK = 1.3VDC.  Scoping it, there is a 2.5VPP 125MHz sinewave.  This is after the MDI link is active.  I believe the 125MHz indicates/confirms that the MDI side successfully linked at 1000

2. SGMII_SOP/N = 550mVPP (this is with it connected to the MAC (server blade) or not.  The SIN/P from the blade is 1VPP (see pics below)

3. With reset button pressed:  LED_0 = 1.95VDC..... LED_1 = .0033VDC.......LED_2 = .0033VDC

After I see MDI link active: LED_0 = 0.10VDC......LED_1 = 2.44VDC.......LED_2 = 0 > 2.44VDC (activity LED flashing)

Here is SOP/N image (~550mVpp).  Seems low but I tried two different EVMs and both show the same and it makes no difference if connected/terminated to SGMII on server blade.  I didn't see in the datasheet what the levels should be on this particular part.

Here is the SIP/N from the server (MAC).  It appears a bit "healthier" at 1VPP.  But of course, I have little experience a this.  I read in another E2E thread that the SGMII spec calls for 675 - 1725mV

Regards,

Phil

Hi Phil,

Can you please check if resistors R24, R26 are populated on the board? Populating these resistors can cause the VOD to be lower.
For measuring VOD of SGMII SOP/SON, can you share the snapshot of the setup? Particularly, I'm looking for connecting to server blade, point where the signal is probed and termination resistance of the probe/oscilloscope.

Checking why activity LED is toggling will give us a lead on where the issue lies. But it's very difficult to understand that without register access.
Do you know a tentative date of when you'll be able to get the MSP430 setup up and running?

--
Regards,
Gokul. 

Gokul,

I'm expecting the Launchpad to arrive this week.  Hopefully it will be easy to get going and allow us to dive into the registers.

Below are some images of my interface and setup.  I didn't have a diff probe to take the measurements so I used two probes w/math function on my Textronix MSO54 scope.  I will look into getting a diff probe.  While my measuring method could be the culprit, the fact that I saw what looked like good signal levels on the TX of the blade (SI_N/P of EVM), gave me some reasonable confidence with using the two probe method. I measured both at the coax solder points on the interface board and also on the EVM board in the area where land patterns for the PU/PD resistors are for the RX_Dx are located.

I have two EVM boards which are configured the same except one has no PU/PD resistors on RX_D0-D3.  The levels on SO_N/P looked essentially the same on both when I swap either of them into the setup.  On the one EVM, I do have R24 and R26 populated.

Unfortunately, I don't have visibility or detailed schematics for the blade's circuitry other than I was told that it is capacitively coupled to a Mellanox IC (MT27528A1).  An engineer knowledgeable on the blade's hardware explained that the interface auto-selects either SGMII or KR and I know for sure that using the Marvell part (88X3540) works fine if I replace TI's EVM with that part's demo board.

I built the edge connector interface board that you see, for the purpose of doing this integration testing.  I have no components or terminations between the coax connectors and the pins on the edge connectors (identified as a SERDES on their pinout diagram). The traces on my board are Z controlled and length matched.  Again, the signal levels on SO N/P (of the DP83867) are very close to the same whether I have them open ended or connected to the blade.

The mfr of the blade actually uses a Broadcom BCM53115S as their interface on their own backplane, but using that part is not an option.

Below is what I'm trying to do

Phil

Hi Phil,

Can you please measure the values of R24,R26?

--
Regards,
Gokul.

Hi Gokul,

The values are 10K.  Again, I have two different DP83867EVM boards.  Both are configured the same except I populated R15, 16, R21 & R24 on one of them as a test.  These are not installed on the other board.  Both boards appear to be outputting very close to the same levels on SO_N/P.  As a matter of fact, I had the one without these parts installed running on the bench just now and was looking at the SO_N/P again.  Measures ~600mVPP. 

I just ran this last test after realizing that there were two series caps on the SGMII signal paths.  One set on the EVM and one set on the server blade.  I removed/jumpered the set on the EVM.  No change and same VPP.  The datasheet says that the SGMII MUST be coupled with 0.1uF.  I'm not sure how critical that value is but two in series would decrease the effective values of those on the EVM.  I will have to pull one off the blade to verify their values.

Below was my reasoning for trying the resistors on the one board.

Phil

Gokul,

Sorry, I misspoke.  On the one board, I populated these per the recommendation under Table 5 (page 37). 

Rhi = 4.02K (R15, 17, 21 & 25)

Rlo = 10K (R16, 19, 24 & 26)  

I verified that the SGMII coupling caps on the server blade are 100nF.

Phil

Hi Phil,

Resistors of 10K wouldn't have changed the VOD. 

Please let me know when the MSP430 setup is available. In the meantime, let me check out a few things from my side on the VOD.

--
Regards,
Gokul.

Gokul,

I have the USB2MDIO app running with a Launchpad and is connected to the EVM.

I've checked several registers and nothing is popping out at me other than SGMII autonegotiate not completing.

0010 = 5848

0011 = AF02

0014 = 297C

0037 = 0002

006E = 8800

I'll keep looking but do you have any thoughts on the next step?

Phil

Gokul,

Here are some other register readings:

0000 = 1140

0001 = 796D

0004 = 01E1

0005 = C1E1

0006 = 006D

000F = 3000

0014 = 29C7  (I had mistyped as 297C in previous message)

0018 = 6150

0031 = 10B1 (this indicates that it is in internal test mode, not set for MODE 3 or 4.  However, during reset, the strap VDC is 0.609V which should set mode 3.  I tried writing 1031 and 1071 to this register but it did not help the issue and the value in reg 0037 was still 0002, indicating SGMII AN not complete)

006E = 8800 (indicates both RGMII and SGMII are enabled.  Not sure if this means anything.  I don't see a hardware strap setting to actually disable RGMII)

006F = 0100

00D3 = 0000

0170 = 0C0E

01D5 = F500

Phil

Hi Phil,

Sorry for the delay in response. I was on PTO and couldn't attend to you in time.

Can you please read the register 0x0037 multiple times and check if the register value is changing?
Can you also provide the read out values of register 0x0038?

You can try swapping the polarity of wires connected to the server blade to check if this makes the PHY link up on the SGMII side.

--
Regards,
Gokul Koraganji.

Hi Gokul,

Reg 0037 is consistently reporting 0002 after multiple reads.

Reg 0038 reports 0020 (I don't see this reg listed in datasheet)

When I swap polarity of the SGMII connections (RX +<>- and TX +<>-), 0037 goes to reporting 0000 and 0038 still reports 0020.  Still no link with blade.  

Phil

Hi Phil,

Can you please provide the read value of 0x004F too with the original polarity?
Also, can you please try inverting only the SOP, SOM polarity and keep the SIP, SIM polarity intact?

You can ignore the read of 0x006E indicating that RGMII mode is enabled. The bit[12] is a reserved bit and has no information.

--
Regards,
Gokul.

Gokul,

With original polarity reg 0004 reads 01E1 (many tries) and reg 0037 went back to reading 0002

When I swap polarity of just SO, still no link and reg 0037 goes to reading 0000.

Phil

Hi Phil,

I meant the read out value of register 0x004F. Can you please provide this?

Can you please try link-up of EVM SGMII to EVM SGMII?
You can make the following connection (EVM1 MDI connected to EVM2 MDI, EVM1 SGMII connected to EVM2 SGMII).

With this experiment, we will get an idea whether the problem is with TI867 alone or the interaction with TI867 and server blade. 

Can you please check if SGMII auto-negotiation can be disabled on server blade side?

--
Regards,
Gokul.

Gokul,

Sorry about that.  0X004F reads 6791

I linked 2 EVMs together as you requested.  0X0037 on first read was 0003, then on subsequent reads it was 0001, indicating SGMII AN successful.  

I have pushed the question about SGMII AN on the blade to the mfr's engineer I have been talking with.  He is in a very distant time zone so it may take a day or so for a response.  However, I suspect the answer would be that it cannot realistically be changed.  The blade is essentially a commodity product with firmware pretty well locked down and very tightly controlled.

Do you think there is any significance to 0X0031 reporting 10B1?  It still does with the two EVMs linked.

Phil

Hi Phil,

Thanks for checking with the server team on the auto-negotiation.
Can you please check with them if they monitor some internal parameters in the blade to check why auto-negotiation is not going through?

I will also check from my side on how to increase the VOD or change some parameters which might make the boards link-up.

Let me check on the read-out value of 0x0031 and get back to you by Monday.

--
Regards,
Gokul.

Hi Gokul,

I'm still awaiting a response from the blade mfr/engineer on the AN question. I will pose this one too.

I'm wondering if I can run a test here by disabling AN on the EVM.  I see the setting for disable but what is not clear is what happens at the copper side or if I would need to do something there, with my copper side link partner.  I'm just wondering if there is some test I can do to force a specific speed at the SGMII.  If AN is not actually enabled on the blade, I suspect it is fixed at 1G.

Phil

Gokul,

For what it's worth, this is a block diagram of what the blade mfr does to interface the SGMII ports to the outside world through their own backplane circuitry.  Unfortunately, I don't have access to any information having to do with settings on these Broadcom chips nor do I have access to their datasheets. The yellow color block is the connection to the blade.  I think they used the xcvr chip on the one port because the switch chip has only one SGMII interface.  Anyway, this is what I am hoping to use the DP 83867 (x2) for.

Phil

Hi Phil,

We can verify the SGMII link with BCM53115S and BCM54618SE to see if the issue persists with both of them.

--
Regards,
Gokul.

Hi Gokul,

Unless you (TI) has access to these parts, I'm not sure how easy testing with the Broadcom parts will be (availability of demo boards, etc) other than I can say that I have an example backplane system from the blade mfr with this Broadcom configuration and it works perfectly.  I have no schematics though.  I will however continue to press the mfr for more info.

Hi Phil,

We don't have access to Broadcom parts. We will have to debug this with the blade mfr with you as the bridge.

Thanks for your support.

--
Regards.
Gokul.