DP83867CR: 1000BASE-T Idle Errors, Data Errors, Connection Issues

Hi Barry,

Do you mind sharing the portion of the schematic with the DP83867CR?

If you do not have the cap divider, you do risk overdriving the internal circuitry, however, the issues you see do not sound like that is the cause.
Have you tried the configuration with the cap coupling and with the correct RBIAS?

Are the switches and PHYs you connect to supporting EEE?

Kind regards,
Ross

Hello, Ross,

Thank you for looking at my issue.  I think I attached the relevant pages of the schematic below.  We have not tested yet with the cap divider and 11k RBIAS.  The rework to add the cap divider will be very delicate.

One PHY we cannot connect to at all is a Qualcomm/Atheros AR8031  If we connect the AR8031 and the DP83867 both to a GigE switch then the two systems can communicate (with errors).

Exhibit_PHY_Sch.pdf

We put the 11kΩ resistors on RBIAS. This had no effect. We also looked carefully at the power supplies. 1.0V and 2.5V both had less than 20mV of noise. We don't supply 1.8V, but we watched the level at a 1.8V pin. It was also very low noise at 1.8V.

We have not yet been able to get the capacitor divider on Clk In. The 25MHz clock only swings to 2.2V and doesn't show any ringing.

We have a previous prototype with much the same circuit but different magnetics and RJ45 connectors. The PHYs on that prototype demonstrate the same 0xFF Idle count immediately after connecting. We can't read the idle count register very quickly, but each time we re-read it is already back to 0xFF.

We have been monitoring the interrupt registers when connected to two different commercial GigE switches.

LINK_STATUS_CHNG_INT, FALSE_CARRIER_INT, and XGMII_ERR_INT are always set even after multiple ISR reads. They clear when physically disconnected from the switches.

Digging in to Link Status; The BMSR LINK STAUS bit is always low, even after multiple reads, but every time we read the PHYSTS LINK_STATUS bit it reads '1'.

Under these conditions we still get mostly reliable data through to the RGMII.

Do any of those results ring a bell?

Still no luck diagnosing our issue.  As I mentioned we have three DP83867 devices per board.  When we connect two on the same board they negotiate and connect without error.  When we try to connect identical PHYs on two separate boards they will not connect at all.  We are looking for clues in the registers.

We are also looking at grounding and the center taps on the magnetics.

I notice the architecture of our magnetics is different than described in the troubleshooting guide.  

Can you offer an opinion on using this transformer arrangement with the DP83867?S558-5999-P3-F.pdf

Hi Barry,

Some thoughts...

Regarding the transformer spec sheet you posted, did you compare the transmission characteristics of it against TI's PHY magnetics guidelines? I'm not sure if they post it on the web, but I believe they will send it to you ( if you don't already have it ).

I just took a quick look at the specs for your Bel transformer. It seemed OK with maybe the exception of the common mode rejection. I think TI is recommending a better rejection ratio at the lower frequency band. Please let me know if you agree and whether you think the difference is important.

If common mode noise is a problem, then maybe you do have an issue with grounding or noise near the transformer & RJ45. If you put a moated ground plane under the analog front end area, then I wonder if you tried grounding it to a metal chassis or tried tweaking the filter caps between your digital and analog/earth ground for the AFE.

Please let us know how you resolve your issue.

Good luck,

Bob

We continue to investigate this connection issue, but have made little progress.

We have two prototype systems that implement the DP83867CR each uses three of the PHYs connected to an FPGA.  We have a few copies of the initial prototype board.

1. When we connect two PHYs on the same board (either board), they connect quickly at 1000BaseT and never produce idle errors.
2. When we connect one PHY on one board to an identical configuration on a second board, they never connect.  Sometimes we see an idle error count.  This might indicate it nearly connected.

We have been trying shielded cables and various shield and ground termination options with no success.  The prototype boards involved in this test use the Bel magnetics I posted earlier.  We are testing with a 1'  unshielded cable, a 5' shielded cable, and a 50' unshielded cable.  The 50' cable might be closest to working, but it still doesn't connect.

We now have very sophisticated capabilities to read and write the PHY registers. 

Can you recommend any tests that might help two DP83867CR devices on separate boards connect to each other?

Hi Barry,

Where is the reference clock coming from for the 2 DP83867 devices? Is the reference clock sourced from the FPGA? In the case of 2 DP83867s connecting together on different boards, are they still sourced from the same FPGA?

Can you apply a reset pulse to only a single DP83867 in that configuration to get the link up?

Can you force the MDI/MDIX configuration on one of the DP83867s in this link pair, and does that cure your link problems?

Best Regards,

Hello, Rob,

We use a 25MHz oscillator connected to the FPGA to generate separate reference clock signals out of the FPGA for the three PHYs. We have even inverted the reference clock to one of the PHYs as a test. It didn't help.

Each board has its own oscillator and FPGA. We have looked at the frequency and jitter of the reference clock signals and haven't seen any issue.

I'll add a point #3: The PHYs on each of these boards can connect two a switch. They both report idle errors, but they both connect to two different switches. I don't know the PHY devices in the switches.

We have written the SW_Reset bit on each of the PHYs many many times to attempt connecting and it does not help. Is there a different "reset pulse" we should try?

We were just now attempting to control MDI/MDIX configuration. Our initial tests are two PHYs on a single board to verify our setup. When we try to control MDI/MDIX on both PHYs using the PHYCR (0x0010) register, the two PHYs refuse to connect. It doesn't matter the MDI configuration of each port. If one or both of the ports have automatic crossover enabled, then the link comes up (for two PHYs on the same board). Does this make any sense? We also noticed the data sheet labels the MDI_CROSSOVER bits as read only. We assume this is a typo.

Thank you very much for looking at this issue.

We found a mistake in our MDIX testing. Setting the bits manually works as expected. We were accidentally shutting down the clock.

Now we will see if it has an effect board-to-board.

We have tried to trim down the PHY capabilities as much as possible.

Master/Slave is configured manually. MDI/MDIX is configure manually. Half Duplex is disabled. Clock out is disabled. RGMII is disabled. 10 and 100 modes are disabled.

With one on one board set to Master + MDI, and the other PHY on the other board set to Slave + MDIX, they still will not connect to each other. They will both still connect to a switch (with idle errors), or to DP83867CR PHYs on the same board (without idle errors).

We do notice that the STS1 register does update when the PHYs on separate boards are connected to each other. STS1 reports the proper values for Master/Slave Configuration Result and Full/Half Duplex capability. When both PHYs are configured as master or both MDI, these values in STS1 are not updated.

It seems to me this indicates the PHYs are exchanging some information, but something is failing before the connection is complete.

Here is a photo of our two early prototype boards to help you envision our setup.  The 4th RJ45 on each board goes to an Atheros 8031 PHY connected to an ARM processor.

We have another test result today based on your question about how the reference clocks are generated.

1. In our previous tests each board used its own 25MHz oscillator.  The PHYs would not connect between boards even if the boards grounds were tied together.
2. We reprogrammed the FPGAs so board A would transmit its 25MHz clock to a test point.  We reprogrammed board B to accept a reference clock on a test point and ignore its oscillator.
3. When PHYs on both boards are using a reference clock from the same oscillator the PHYs connect without error.

You can see in the picture, the clock signal integrity is far from ideal, but it works better than the independent oscillator (which you can see right next to the test points).

This leads me to the conclusion the issue is not magnetic, termination, or power related.  It seems to be some sort of reference clock problem.

We are going to try some different clocking options in the FPGA to see if we can learn more.

I still welcome any insight or recommendations.

No luck adjusting the reference clock.  We took the 25MHz oscillator signal through an FPGA PLL then to registered FPGA pins to clean it or at least change it.  No change in PHY connectivity.

We also finally did the capacitor divider rework on the ref clock signal.  We can see the clock voltage is below 1V pp, but again no improvement in connectivity.

The DP83867CR does not specify much about the external clock.  We are using a mems oscillator with 25ppm accuracy (datasheet attached).  Is there any reason to suspect the oscillator?

I am going to try another rework to connect the oscillator output directly to the PHY through a cap divider.

Any other suggestions?

Do most users just connect crystals?  Is it rare to try and drive XI with a clock?

20005624B.pdf

Thanks, Rob,

That makes sense and seems to agree with what we are seeing. Can you provide any better detail on the jitter requirement for XI? I assume the 1.2ps is RMS. Do you have a peak to peak limit?

It is not mentioned at all in the datasheet. It's a little surprising that level of performance is required without any notice.

We replace our oscillator with the SiTime SIT8208AC-23-25E-25.000000X

It specifies RMS Period Jitter Max at 2 ps which is still larger than the 1.2ps recommended in the previous post.  Our other oscillator was 9 ps max.

The new oscillator at least lets us connect reliably to switches.   I could not find any oscillators in any package that promised less than 2ps RMS jitter.  The only way I know to generate a lower jitter clock is with a complex clock generator.

Our plan is to rev the board and use individual crystals for each PHY.  It seems to me this should probably be a data sheet / app note recommendation.

Thanks for the help, I think we are moving forward again.

Hi Barry,

You may be alright with the RMS period jitter of 2ps if the frequency is within the +/- 50ppm accuracy requirement and your design passes IEEE compliance for TX jitter.

Thank you for the suggestion on the reference clock. We are planning an app note on reference clock design/selection for the DP83867 device.

Best Regards,