DP83867IS: errors in data transfer, SGMII interface

Bart

Part Number: DP83867IS

Team,

we are trying to solve an issue with DP83867IS, connected to Marvell switch.

It is connected with SGMII interface and the link from PHY to switch shows errors in data transfer. It has different effects on the four connected lines/PHYs. On two lines some packets are lost, on other two SGMII autonegotiation does not pass at all. With disabled autonegotiation the transfer works, but with errors. The opposite direction from switch to PHY works well.

There is only 100n series capacitor connected on the link and the PCB is differential routed at 100 ohms (50 ohm single impedance). I have tried to add series termination resistor, and lower the value of capacitor but with no effect.

When looking at bad received data (attached below is transmitted and received packet), there is always 5 lower bits in a byte negative. At first look it seems like the data is disarranged. This forces me the idea there is some bit error in 8/10 coding (which uses 3 and 5 bits coding). The received data are the same at both ports and on two different borads.

Have you encountered this behavior previously on other designs? Do you have any solutions to recommend?

Thanks for your help,

Bartosz

Fullscreen ping-s1000-tx.txt Download

root@h2:~# ping -c 1 -b -s 1000 192.168.1.255
WARNING: pinging broadcast address
PING 192.168.1.255 (192.168.1.255) 1000(1028) bytes of data.
21:44:55.738423 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 1028)
    192.168.1.2 > 192.168.1.255: ICMP echo request, id 1469, seq 1, length 1008
	0x0000:  ffff ffff ffff 001b 217b 2d64 0800 4500  ........!{-d..E.
	0x0010:  0404 0000 4000 4001 b2a7 c0a8 0102 c0a8  ....@.@.........
	0x0020:  01ff 0800 8b48 05bd 0001 c7e6 a65a 6844  .....H.......ZhD
	0x0030:  0b00 0809 0a0b 0c0d 0e0f 1011 1213 1415  ................
	0x0040:  1617 1819 1a1b 1c1d 1e1f 2021 2223 2425  ...........!"#$%
	0x0050:  2627 2829 2a2b 2c2d 2e2f 3031 3233 3435  &'()*+,-./012345
	0x0060:  3637 3839 3a3b 3c3d 3e3f 4041 4243 4445  6789:;<=>?@ABCDE
	0x0070:  4647 4849 4a4b 4c4d 4e4f 5051 5253 5455  FGHIJKLMNOPQRSTU
	0x0080:  5657 5859 5a5b 5c5d 5e5f 6061 6263 6465  VWXYZ[\]^_`abcde
	0x0090:  6667 6869 6a6b 6c6d 6e6f 7071 7273 7475  fghijklmnopqrstu
	0x00a0:  7677 7879 7a7b 7c7d 7e7f 8081 8283 8485  vwxyz{|}~.......
	0x00b0:  8687 8889 8a8b 8c8d 8e8f 9091 9293 9495  ................
	0x00c0:  9697 9899 9a9b 9c9d 9e9f a0a1 a2a3 a4a5  ................
	0x00d0:  a6a7 a8a9 aaab acad aeaf b0b1 b2b3 b4b5  ................
	0x00e0:  b6b7 b8b9 babb bcbd bebf c0c1 c2c3 c4c5  ................
	0x00f0:  c6c7 c8c9 cacb cccd cecf d0d1 d2d3 d4d5  ................
	0x0100:  d6d7 d8d9 dadb dcdd dedf e0e1 e2e3 e4e5  ................
	0x0110:  e6e7 e8e9 eaeb eced eeef f0f1 f2f3 f4f5  ................
	0x0120:  f6f7 f8f9 fafb fcfd feff 0001 0203 0405  ................
	0x0130:  0607 0809 0a0b 0c0d 0e0f 1011 1213 1415  ................
	0x0140:  1617 1819 1a1b 1c1d 1e1f 2021 2223 2425  ...........!"#$%
	0x0150:  2627 2829 2a2b 2c2d 2e2f 3031 3233 3435  &'()*+,-./012345
	0x0160:  3637 3839 3a3b 3c3d 3e3f 4041 4243 4445  6789:;<=>?@ABCDE
	0x0170:  4647 4849 4a4b 4c4d 4e4f 5051 5253 5455  FGHIJKLMNOPQRSTU
	0x0180:  5657 5859 5a5b 5c5d 5e5f 6061 6263 6465  VWXYZ[\]^_`abcde
	0x0190:  6667 6869 6a6b 6c6d 6e6f 7071 7273 7475  fghijklmnopqrstu
	0x01a0:  7677 7879 7a7b 7c7d 7e7f 8081 8283 8485  vwxyz{|}~.......
	0x01b0:  8687 8889 8a8b 8c8d 8e8f 9091 9293 9495  ................
	0x01c0:  9697 9899 9a9b 9c9d 9e9f a0a1 a2a3 a4a5  ................
	0x01d0:  a6a7 a8a9 aaab acad aeaf b0b1 b2b3 b4b5  ................
	0x01e0:  b6b7 b8b9 babb bcbd bebf c0c1 c2c3 c4c5  ................
	0x01f0:  c6c7 c8c9 cacb cccd cecf d0d1 d2d3 d4d5  ................
	0x0200:  d6d7 d8d9 dadb dcdd dedf e0e1 e2e3 e4e5  ................
	0x0210:  e6e7 e8e9 eaeb eced eeef f0f1 f2f3 f4f5  ................
	0x0220:  f6f7 f8f9 fafb fcfd feff 0001 0203 0405  ................
	0x0230:  0607 0809 0a0b 0c0d 0e0f 1011 1213 1415  ................
	0x0240:  1617 1819 1a1b 1c1d 1e1f 2021 2223 2425  ...........!"#$%
	0x0250:  2627 2829 2a2b 2c2d 2e2f 3031 3233 3435  &'()*+,-./012345
	0x0260:  3637 3839 3a3b 3c3d 3e3f 4041 4243 4445  6789:;<=>?@ABCDE
	0x0270:  4647 4849 4a4b 4c4d 4e4f 5051 5253 5455  FGHIJKLMNOPQRSTU
	0x0280:  5657 5859 5a5b 5c5d 5e5f 6061 6263 6465  VWXYZ[\]^_`abcde
	0x0290:  6667 6869 6a6b 6c6d 6e6f 7071 7273 7475  fghijklmnopqrstu
	0x02a0:  7677 7879 7a7b 7c7d 7e7f 8081 8283 8485  vwxyz{|}~.......
	0x02b0:  8687 8889 8a8b 8c8d 8e8f 9091 9293 9495  ................
	0x02c0:  9697 9899 9a9b 9c9d 9e9f a0a1 a2a3 a4a5  ................
	0x02d0:  a6a7 a8a9 aaab acad aeaf b0b1 b2b3 b4b5  ................
	0x02e0:  b6b7 b8b9 babb bcbd bebf c0c1 c2c3 c4c5  ................
	0x02f0:  c6c7 c8c9 cacb cccd cecf d0d1 d2d3 d4d5  ................
	0x0300:  d6d7 d8d9 dadb dcdd dedf e0e1 e2e3 e4e5  ................
	0x0310:  e6e7 e8e9 eaeb eced eeef f0f1 f2f3 f4f5  ................
	0x0320:  f6f7 f8f9 fafb fcfd feff 0001 0203 0405  ................
	0x0330:  0607 0809 0a0b 0c0d 0e0f 1011 1213 1415  ................
	0x0340:  1617 1819 1a1b 1c1d 1e1f 2021 2223 2425  ...........!"#$%
	0x0350:  2627 2829 2a2b 2c2d 2e2f 3031 3233 3435  &'()*+,-./012345
	0x0360:  3637 3839 3a3b 3c3d 3e3f 4041 4243 4445  6789:;<=>?@ABCDE
	0x0370:  4647 4849 4a4b 4c4d 4e4f 5051 5253 5455  FGHIJKLMNOPQRSTU
	0x0380:  5657 5859 5a5b 5c5d 5e5f 6061 6263 6465  VWXYZ[\]^_`abcde
	0x0390:  6667 6869 6a6b 6c6d 6e6f 7071 7273 7475  fghijklmnopqrstu
	0x03a0:  7677 7879 7a7b 7c7d 7e7f 8081 8283 8485  vwxyz{|}~.......
	0x03b0:  8687 8889 8a8b 8c8d 8e8f 9091 9293 9495  ................
	0x03c0:  9697 9899 9a9b 9c9d 9e9f a0a1 a2a3 a4a5  ................
	0x03d0:  a6a7 a8a9 aaab acad aeaf b0b1 b2b3 b4b5  ................
	0x03e0:  b6b7 b8b9 babb bcbd bebf c0c1 c2c3 c4c5  ................
	0x03f0:  c6c7 c8c9 cacb cccd cecf d0d1 d2d3 d4d5  ................
	0x0400:  d6d7 d8d9 dadb dcdd dedf e0e1 e2e3 e4e5  ................
	0x0410:  e6e7                                     ..

Fullscreen ping-s1000-rx.txt Download

root@h1:~# tcpdump -i cst0 -n -XXX -vvv
tcpdump: listening on cst0, link-type EN10MB (Ethernet), capture size 65535 bytes
21:42:57.033636 ce:cd:cc:cb:ca:c9 > d4:d3:d2:d1:cf:d0, ethertype Unknown (0xd7d8), length 957: 
	0x0000:  d4d3 d2d1 cfd0 cecd cccb cac9 d7d8 c6c5  ................
	0x0010:  dbc3 ddde dfa0 a1a2 bca4 bab9 a7a8 b6b5  ................
	0x0020:  b4b3 b2b1 afb0 aead acab aaa9 b7b8 a6a5  ................
	0x0030:  bba3 bdbe bf60 6162 7c64 7a79 6768 7675  .....`ab|dzyghvu
	0x0040:  7473 7271 6f70 6e6d 6c6b 6a69 7778 6665  tsrqopnmlkjiwxfe
	0x0050:  7b63 7d7e 7f80 8182 9c84 9a99 8788 9695  {c}~............
	0x0060:  9493 9291 8f90 8e8d 8c8b 8a89 9798 8685  ................
	0x0070:  9b83 9d9e 9f40 4142 5c44 5a59 4748 5655  .....@AB\DZYGHVU
	0x0080:  5453 5251 4f50 4e4d 4c4b 4a49 5758 4645  TSRQOPNMLKJIWXFE
	0x0090:  5b43 5d5e 5f20 2122 3c24 3a39 2728 3635  [C]^_.!"<$:9'(65
	0x00a0:  3433 3231 2f30 2e2d 2c2b 2a29 3738 2625  4321/0.-,+*)78&%
	0x00b0:  3b23 3d3e 3fe0 e1e2 fce4 faf9 e7e8 f6f5  ;#=>?...........
	0x00c0:  f4f3 f2f1 eff0 eeed eceb eae9 f7f8 e6e5  ................
	0x00d0:  fbe3 fdfe ff00 0102 1c04 1a19 0708 1615  ................
	0x00e0:  1413 1211 0f10 0e0d 0c0b 0a09 1718 0605  ................
	0x00f0:  1b03 1d1e 1fc0 c1c2 dcc4 dad9 c7c8 d6d5  ................
	0x0100:  d4d3 d2d1 cfd0 cecd cccb cac9 d7d8 c6c5  ................
	0x0110:  dbc3 ddde dfa0 a1a2 bca4 bab9 a7a8 b6b5  ................
	0x0120:  b4b3 b2b1 afb0 aead acab aaa9 b7b8 a6a5  ................
	0x0130:  bba3 bdbe bf60 6162 7c64 7a79 6768 7675  .....`ab|dzyghvu
	0x0140:  7473 7271 6f70 6e6d 6c6b 6a69 7778 6665  tsrqopnmlkjiwxfe
	0x0150:  7b63 7d7e 7f80 8182 9c84 9a99 8788 9695  {c}~............
	0x0160:  9493 9291 8f90 8e8d 8c8b 8a89 9798 8685  ................
	0x0170:  9b83 9d9e 9f40 4142 5c44 5a59 4748 5655  .....@AB\DZYGHVU
	0x0180:  5453 5251 4f50 4e4d 4c4b 4a49 5758 4645  TSRQOPNMLKJIWXFE
	0x0190:  5b43 5d5e 5f20 2122 3c24 3a39 2728 3635  [C]^_.!"<$:9'(65
	0x01a0:  3433 3231 2f30 2e2d 2c2b 2a29 3738 2625  4321/0.-,+*)78&%
	0x01b0:  3b23 3d3e 3fe0 e1e2 fce4 faf9 e7e8 f6f5  ;#=>?...........
	0x01c0:  f4f3 f2f1 eff0 eeed eceb eae9 f7f8 e6e5  ................
	0x01d0:  fbe3 fdfe ff00 0102 1c04 1a19 0708 1615  ................
	0x01e0:  1413 1211 0f10 0e0d 0c0b 0a09 1718 0605  ................
	0x01f0:  1b03 1d1e 1fc0 c1c2 dcc4 dad9 c7c8 d6d5  ................
	0x0200:  d4d3 d2d1 cfd0 cecd cccb cac9 d7d8 c6c5  ................
	0x0210:  dbc3 ddde dfa0 a1a2 bca4 bab9 a7a8 b6b5  ................
	0x0220:  b4b3 b2b1 afb0 aead acab aaa9 b7b8 a6a5  ................
	0x0230:  bba3 bdbe bf60 6162 7c64 7a79 6768 7675  .....`ab|dzyghvu
	0x0240:  7473 7271 6f70 6e6d 6c6b 6a69 7778 6665  tsrqopnmlkjiwxfe
	0x0250:  7b63 7d7e 7f80 8182 9c84 9a99 8788 9695  {c}~............
	0x0260:  9493 9291 8f90 8e8d 8c8b 8a89 9798 8685  ................
	0x0270:  9b83 9d9e 9f40 4142 5c44 5a59 4748 5655  .....@AB\DZYGHVU
	0x0280:  5453 5251 4f50 4e4d 4c4b 4a49 5758 4645  TSRQOPNMLKJIWXFE
	0x0290:  5b43 5d5e 5f20 2122 3c24 3a39 2728 3635  [C]^_.!"<$:9'(65
	0x02a0:  3433 3231 2f30 2e2d 2c2b 2a29 3738 2625  4321/0.-,+*)78&%
	0x02b0:  3b23 3d3e 3fe0 e1e2 fce4 faf9 e7e8 f6f5  ;#=>?...........
	0x02c0:  f4f3 f2f1 eff0 eeed eceb eae9 f7f8 e6e5  ................
	0x02d0:  fbe3 fdfe ff00 0102 1c04 1a19 0708 1615  ................
	0x02e0:  1413 1211 0f10 0e0d 0c0b 0a09 1718 0605  ................
	0x02f0:  1b03 1d1e 1fc0 c1c2 dcc4 dad9 c7c8 d6d5  ................
	0x0300:  d4d3 d2d1 cfd0 cecd cccb cac9 d7d8 c6c5  ................
	0x0310:  dbc3 ddde dfa0 a1a2 bca4 bab9 a7a8 b6b5  ................
	0x0320:  b4b3 b2b1 afb0 aead acab aaa9 b7b8 a6a5  ................
	0x0330:  bba3 bdbe bf60 6162 7c64 7a79 6768 7675  .....`ab|dzyghvu
	0x0340:  7473 7271 6f70 6e6d 6c6b 6a69 7778 6665  tsrqopnmlkjiwxfe
	0x0350:  7b63 7d7e 7f80 8182 9c84 9a99 8788 9695  {c}~............
	0x0360:  9493 9291 8f90 8e8d 8c8b 8a89 9798 8685  ................
	0x0370:  9b83 9d9e 9f40 4142 5c44 5a59 4748 5655  .....@AB\DZYGHVU
	0x0380:  5453 5251 4f50 4e4d 4c4b 4a49 5758 4645  TSRQOPNMLKJIWXFE
	0x0390:  5b43 5d5e 5f20 2122 3c24 3a39 2728 3635  [C]^_.!"<$:9'(65
	0x03a0:  3433 3231 2f30 2e2d 2c2b 2a29 3738 2625  4321/0.-,+*)78&%
	0x03b0:  3b23 3d3e 3fe0 e1e2 fce4 faf9 e7         ;#=>?........

over 8 years ago

0 Bart over 8 years ago

TI__Expert 6500 points

Team,

any feedback on this?

Regards

0 Rob Rodrigues over 8 years ago in reply to Bart

TI__Mastermind 19780 points

Hi Bartosz,

First thing to check is that RX_CTRL pin is strapped to mode 3. Please ENSURE RX_CTRL is in mode 3.

Can you provide a schematic of the PHYs that are working, and the PHYs that are not? From my understanding, you have 4 PHYs in this system.

Please also give me the top markings from the DP83867IS units on the board.

Best Regards,

0 Radek Jahoda over 8 years ago in reply to Rob Rodrigues

Prodigy 85 points

Hello Rob,

thank you for your answer. The RX_CTLR pin is in MODE 3. Previously we decided to put it to MODE 4 to disable autonegotiation, but we have enabled it after finding, that it cannot be enabled by software, if the strap pin is in MODE 4.

There are 4 PHYs in the design and 2 switches Marvell 88E6390X. PHYs 1 and 3 are connected to switch 1, PHYs 2 and 4 are connected to switch 2. Connection is made with SGMII interface. None of the PHYs work 100% correct, but sending packets from switch to PHY works always OK, but in the oposite direction from PHY to switch:

- PHYs 1 and 3 connected to switch 1 do not autonegotiate SGMII at all, but setting them without autonegotiation will allow to send packets, but all packets are lost (the oposite direction works fine)

- PHYs 2 and 4 autonegotiate SGMII, but some packets are errored in this direction (guess about 1%)

All PHYs use the same schematic except setting of SMI address. The other difference is in routing, where SGMII connection to PHYs 1 and 3 has 3 vias (the not working ones), while connection to PHYs 2 and 4 has only 2 vias. But the SGMII link is routed very carefully 100 ohms differencially, 50 ohms single impedance. I can send the schematic but currently is seems the forum can't allow me to insert the attachment.

Is there some way to set drive strength/termination in the PHY?

Top markings are:

TI83867IS

TI 7AI

CDYX G4

Thank you very much for your feedback.

Best regards

Radek

0 Rob Rodrigues over 8 years ago in reply to Radek Jahoda

TI__Mastermind 19780 points

Hi Radek,

I have sent you a friend request so you can share your schematic with me privately.

For the SMI address straps, did you place dummy resistors to balance the impedance on the SGMII traces? For example, if you place a mode 2 strap on pin 35(SGMII_SOP), then a mode 2 strap should be placed on pin 36(SGMII_SON) as well.

Is your 25MHz reference clock for the PHYs coming from the switch or a crystal?

How close to the PHY are the DC blocking caps of PHY 2 & 4 compared to PHY 1 & 3? This can sometimes cause reflections that significantly disturb signal integrity.

You can adjust the IO impedance of the drivers using register 0x0170 bit[4:0].

Best Regards,

0 Radek Jahoda over 8 years ago in reply to Rob Rodrigues

Prodigy 85 points

Thank you, I have sent you the schematic by private message.

Yes, there are the same strap resistors on SGMI_SOP/SON and SGMI_SIP/SIN.
The reference clock is from one external clock oscillator, the clock is then buffered for each PHY and Switch independently.
The blocking capacitors are at the same positions on all PHYs, directly on the pins.
We will try to experiment with the IO impedance and I will post the results.
Thank you
Radek

Interface

Interface forum

DP83867IS: errors in data transfer, SGMII interface