DP83867CR: RX_CTRL Signal Not Sampled or Shifted Correctly on FPGA

Hi Asmir,

We are looking into this issue and will provide additional feedback by Tuesday of next week.

Is this phenomenon causing any packet errors/lost packets? Is communication still clean?

Thank you,

Nikhil

All information in this correspondence and in any related correspondence is provided “as is” and “with all faults”, and it is subject to TI’s Important Notice (http://www.ti.com/corp/docs/legal/important-notice.shtml).

Hi Nikhil,

Thank you for your response.

Yes my packet generator reports a lot of FCS errors with higher link utilization. When the link utilization drops there are no errors.

Also as said if all frames have even number of bytes there is no problems even with 100% link utilization.

I can provide actual numbers (frame drop rate with different utilization and frame sizes) if needed.

Asmir

Hi Asmir,

Can you provide a register dump of the following registers for the case where an even number of bytes is used at 100% utilization vs an odd number of bytes at 100% utilization

• 0x0 - 0x1F
• 0x2D - 0x2E
• 0x6E - 0x6F 
• 0x31 - 0x33
• 0x86
• 0x170

Thank you,

Nikhil

All information in this correspondence and in any related correspondence is provided “as is” and “with all faults”, and it is subject to TI’s Important Notice (http://www.ti.com/corp/docs/legal/important-notice.shtml).

Hi Nikhil, 

Thanks for reaching out.

I've dumped the required registers, here is a link to the sheet, I hope the format is acceptable.

reg_dump_spredsheet

I've checked output for sensibility but feel free to request additional tests with other conditions if needed.

Asmir

Hi Asmir,

Unfortunately I am being blocked from the link attached. Are you able to upload register dump as an excel file?

What is the format of the link attached (dropbox link, google doc link, etc.)?

Thank you,

Nikhil

Oh, It was google docs, I even made sure it's publicly availabe, but who knows, anyway here is another link, I hope this one works.

dp83867_reg

Thanks for your patience.

Asmir

Hi Asmir,

I have sent you a request to connect and have reached out private message. Please check your inbox and I look forward to your reply!

Thank you,

Nikhil

dp83867cr_reg_dump_xls.xls

Message attachment in case of firewall blocking external sites.

Hi Asmir,

Thank you for providing the register values. I will review spreadsheet and provide additional feedback tomorrow.

Thank you,

Nikhil 

Hi Asmir,

When you say you are using an odd size frame, does this mean packet is not byte boundary aligned?

Can you try toggling the odd nibble detect bit in register 0x43?

Thank you,

Nikhil

Hi Nikhil,

I think packets are byte boundary aligned, just their size is different. For example when I generate packets with sizes of 64, 68, 1518, 4000... bytes - everything works.

But with packet sizes of 65, 71, 1519 - I get errors pictured in first post.

I'll try toggling the register field you mentioned and post the results.

Thank you,

Asmir A

Hi Asmir,

Please let me know the status after adjusting the odd nibble detect register value.

Thank you,

Nikhil

All information in this correspondence and in any related correspondence is provided “as is” and “with all faults”, and it is subject to TI’s Important Notice (http://www.ti.com/corp/docs/legal/important-notice.shtml).

Hi Nikhil,

I didn't notice any change after seting odd nibble detect register field. As I can not find any affect the register change has please inform me if you need any register dump after the change.

Thank you.

Asmir A

Hi Asmir,

Thanks for the update. I will look into this issue and provide an update by Friday of this week.

Thank you,

Nikhil

Hi Asmir,

Quick question: What is value of register 0x32 in both cases odd and even byte size?

From a PHY perspective, for 1G data, as long as the packets are byte aligned, PHY should be immune to odd vs even byte length. An IPG of at least 12 bytes (~96ns) is needed for clock recovery in 1G mode. 

What is the link partner used during these tests? I understand and FPGA and DP83867 is used on one side, what is on the other end of the cable? Is it another DP83867 and FPGA setup, and if so, is this second FPGA the one counting errors?

Can the PHY be placed in MII or digital loopback such that the FPGA sends data to the PHY via RGMII, and the PHY loops data back to the FPGA? Is communication clean in this test case?

Thank you,

Nikhil

All information in this correspondence and in any related correspondence is provided “as is” and “with all faults”, and it is subject to TI’s Important Notice (http://www.ti.com/corp/docs/legal/important-notice.shtml).

Hi Nihkil,

I'll get the 0x32 redings ASAP.

To give a more detailed overview, I'm building an ethernet tap with aggregation support. There are four DP83867 phys connected to the FPGA.

But the issue is on the passthrough mode given by the block diagram below, so I'll focus on it.

Traffic from first port is sent to the second and vice versa.

The link partner to the all DP83867 PHYs is Xena Valkyrie traffic generator and analysis platform. So I'm using traffic generator to send data to one port and, when working correctly, should receive the same data from second port.

But when I send packets with odd number of bytes to the first port I receive a lot of FCS errors from the second port when the gigabit link utilization is about 80% and more. Error rates increase with link speed utilization.

I've isolated the issue to be on the receiving end of the FPGA, i.e. packets sent from FPGA to traffic generator/analyzer via DP83867 are all fine, regarding of frame size and link speed.

In contrast packets with uneven number of bytes sent from packet generator to the FPGA are not received correctly and such damaged packets forwarded to the second port give FCS errors on the network traffic analyzer.

What I did more is to isolate one port and attach in-chip logic analyzer to it, so packet generator was sending data to the FPGA and I was sampling the same packets using logic analyzer, where I saw the issue presented on the pictures in the first post.

Simplified by ignorring rest of the system, the issue is receiving data from PHY chip to the FPGA.

I've tested the traffic generator by simply connecting both ports with a cable - there was no issue. Alternetive PHYs in same configuration also don't have any issue.

I'll also try to put PHY into loopback mode and post results (it will take a bit) - thanks for the suggestion.

Best regards, Asmir

Hi Asmir,

So the way I understand this is the Xena traffic generator sends data to the RJ45 of one DP83867 -> Xilinx FPGA -> second DP83867 -> back to Xena. If we can make the below configuration, set the first DP83867 for reverse loopback, such that the Xena sends data to the first DP83867, and the first DP83867 sends it right back to the Xena, can the Xena count packet errors? Are the errors still present? 

Please let me know if you can't view the picture pasted above. Please let me know if further clarification is needed for this setup. 

What is the cable length between Xena and DP83867?

Thank you,

Nikhil Menon

All information in this correspondence and in any related correspondence is provided “as is” and “with all faults”, and it is subject to TI’s Important Notice (http://www.ti.com/corp/docs/legal/important-notice.shtml).

Hi Nikhil,

Yes, you understood right, I'll create suggested configuration shortly and post results.

Regarding MDIO register 0x32, the value I got is 0xD3. Though I've tried various delay setings using 0x32 and 0x86 registers before with 0.25ns steps - results were same or much worse depending on the delay amount. I've also tried with internal FPGA delay with ~80ps resolution - it did not mitigate the issue.

Thank you for your sugestion, I'll post results shortly.

PS: The cable length is no more than 3m, and I tested the Xena Valkyrie by connecting the both ports i.e just sending data from one port to the other without my device between.

Asmir A

Hi, Nikhil

So I've put the PHY into the Far-End (Reverse) loopback mode and disabled FPGA passthrough logic, I was using one port at the time and l think loopback was configured correctly as I got packets back to the generator.

Again with even frames there was no error, but with odd number of bytes I got errors.

Testing was done at 1Gbit, and the latency was around 300ns (if I substracted packet generator latency correctly).

Let me know if you need more data or testing with different loopback mode.

Thank you,

Asmir A

Hi Asmir,

Does the FPGA have the capability to generate packets? Can we try an experiment where we have the FPGA generate packets to PHY 1 via RGMII, connect PHY 1 to PHY 2 so packets are returned back to the FPGA? Will the FPGA be able to check for errors? This setup removes the Xena device from the system, so we can check the PHY and FPGA.

Thank you,

Nikhil

All information in this correspondence and in any related correspondence is provided “as is” and “with all faults”, and it is subject to TI’s Important Notice (http://www.ti.com/corp/docs/legal/important-notice.shtml).

Hi Nikhil,

I've additionaly tested other 3 DP83867 phys on the board in reverse loopback mode - all had the same issue.

I would rule out the packet generator as it was tested on other similar devices and networks switches but I'll try to create the same functionality with a PC network card just to be sure since creating it in FPGA would take more time for a realiable design.

Also as a next step I'm thinking on ordering DP83867ERGZ-R-EVM evaluation board and do the same loopback testing with the goal of ruling out PCB board and components around the PHY chip.

Can you please confirm that the DP83867ERGZ-R-EVM board would be a valid reference for DP83867CR since the PHY manufacturer number is not exactly the same?

I also have to note that currently I'm using DP83867CR in mirror mode, configured with strap pins, because I didn't want to cross any analog signal lines from RJ-45 connector - if that is relevant.

Thank you,

Asmir A

Hi Asmir,

DP83867ERGZ-R-EVM would be a good reference to eliminate any board level issues. Enabling mirror mode should not affect PHY performance. 

If possible, can the loopback test be tried between the EVM and Xena packet generator without connecting anything to the MAC side? This will completely isolate the PHY.

Thank you,

Nikhil

All information in this correspondence and in any related correspondence is provided “as is” and “with all faults”, and it is subject to TI’s Important Notice (http://www.ti.com/corp/docs/legal/important-notice.shtml).

Hi Nikhil,
I'll post results as soon as the board arrives, thank you.

Asmir A

Hi Asmir,

Thanks for the update, please keep me in the loop.

Thank you,

Nikhil

All information in this correspondence and in any related correspondence is provided “as is” and “with all faults”, and it is subject to TI’s Important Notice (http://www.ti.com/corp/docs/legal/important-notice.shtml).