Linux/TLK10232: TLK10232 Link Detection Issue

Avinash N,

I am reviewing your writes, my initial questions:
Is there at least one clock cycle between the first and second writes to allow the device to come out of reset?
In the third write (0x07.0x0), it seems you have a extra 0 at the LSB, are you trying to write 0x2000?
Could you try 0x3000 to enable Auto Negotiation? It seems that Link Training may be failing in this case.
Could you try mdio write 0x01.0x0096 0x0002, to "Enable start-up protocol as per 10GBASE-KR standard" ?

As I am not fimilar with your application is there a reason you disbaled Auto Negotiation and Link Training?

Hi Sir,

         1. Is there at least one clock cycle between the first and second writes to allow the device to come out of reset?

             Yes Sir, after writing we perform a read operation. The value obtained from the register is 0x610.

 2. In the third write (0x07.0x0), it seems you have a extra 0 at the LSB, are you trying to write 0x2000?

     During the Design of hardware they have posted a quiery to TI. In that they have recommended to disable Auto-Negotiation and to disable Link Training. Link for your reference https://e2e.ti.com/support/interface/f/138/t/668892

3.  Could you try 0x3000 to enable Auto Negotiation? Could you try mdio write 0x01.0x0096 0x0002, to "Enable start-up protocol as per 10GBASE-KR standard" ?

     we have tried as per your suggestion. But still the Link id Disabled.

We need to tune any other register ?

Where exactly the issue can be ?

Regards,

Avinash N

Avinash N,

mdio write 0x1E.0x8020 0x03FF: should this register address be 8021? there is no register address 8020

What is the value of Register Address:0x0010 during your test? 

Hi Malik Barton,

                        I have changed the register value as per your suggestion. But still the Link is in Down. I want to know what will be the CLOCK-OUT frequency ? Because we have probed the Input clock (156.25 MHz) we are obtaining it. But when we probed both the CLOCK-OUT of TLK10232 we can able to get exact clock value. What will be the default frequency at CLOCK-OUT pin of TLK10232 ?

How to debug and resolve this issue ?

Thanks in Advance for your support.

Regards,

Avinash N

Avinash N,

CLOCK-OUT frequency will depend on your configuration of the registers below and implementation of TLK10232. By default CLOCK-OUT pins output the high speed side Channel A recovered byte clock (high speed line rate divided by 16 or 20). What are your configurations for these registers:

0x1E.0x0002 Bit [6]: HS_VRANGE
0x1E.0x000D Bit [6]: CLKOUT_DIV[3:0]
0x1E.0x000D Bit [6]: CLKOUT_SEL[3:0]

Below are some steps you can follow for getting a stable connection between TLK10232 and your SFP+ module. I was mistaken with register 0x8020 which should be changed to 0x3ff my apologizes. Some step you already complete which is good but please confirm you are following these steps in your SW.

1. Reset device (write a 1 to 0x1E.0000 bit 15 or assert RESET_N pin)
2. Make sure the reference clock selection (156.25 MHz or 312.5 MHz) is correct – this is done through register 0x1E.001D bit 12 (default is 156.25 MHz).
3. Disable auto-negotiation by writing 1’b0 to 0x07.0000 bit 12
4. Disable link training by writing 16’h0000 to 0x01.0096
5. Write 16’h03FF to 0x1E.8020. This allows the link settings that would normally be configured through KR training to be configured manually instead.
6. Depending on the link conditions, you may need to change the default configuration of 0x1E.0003 and 0x1E.0004. For optical connections, we typically recommend changing HS_ENTRACK (0x1E.0004 bit 15) to 1’b1 and HS_EQPRE (0x1E.0004 bits 14:12) to 3’b101. This can be a starting point, but you may need to do some BER testing to optimize the values.
7. Issue a data path reset by writing 1’b1 to 0x1E.000E bit 3.

Step 6 is a very important step in the process and you should monitor 0x1E.0x0010 for the error count. This allows you to measure your BER which should be minimized for a good link.

Hi Malik Barton,

                        We have configured the registers to access the Channel B, as we have hardwire the PORTAD0 - 1 and the register configuration are as shown below,

                        0x1E.0x0002 Bit [6]: HS_VRANGE - 0 = VCO runs at higher end of frequency range (Default 1’b0)

0x1E.0x000D Bit [6]: CLKOUT_DIV[3:0]  - 1000 = Divide by 4 (Default 4'b1000)

0x1E.0x000D Bit [6]: CLKOUT_SEL[3:0] - 10x0 = Selects Ch B HS recovered byte clock as output clock.

Input Reference clock - 156.25MHz.

And we have performed the Register access sequence as shown below,

write 0x1E.0x0 0x8610

write 0x1E.0x001D 0x0000

write 0x07.0x0 0x20000

write 0x01.0x0096 0x0000

write 0x1E.0x8020 0x03FF

write 0x1E.0x0004 0x9500

write 0x1E.0x0004 0xD500

write 0x1E.0xE 0x0008

After that we read back the Error Counter register(0x1E.0x0010) - 0x0000. 

Question:

1. What will be the output clock at CLOCK_OUT PIN ?

2. Which register need to be checked for Link UP?

3. How to confirm where the issue is exactly? (T4240 Processor <---> TLK10232 <---> SFP+, i.e Through XAUI)

 

Thanks in Advance for your support.

Regards,

Avinash N

 

Avinash N,

1. What will be the output clock at CLOCK_OUT PIN ?

   A) Since 0x1E.0x000D Bit [6]: CLKOUT_SEL[3:0] - 10x0 = Selects Ch B HS recovered byte clock as output clock, then the output clock frequecny should be the Ch B data rate divided by 4.

2. Which register need to be checked for Link UP?

   A) Please read 0x03.0x0001 and 0x0008 for link status. Also see PCS_RX_LINK_STATUS in data manual.  Also PCS_RX_LINK_STATUS (Address 0x03, 0x0020, bit12) can be set to show link up on HS side. This bit indicates that auto-negotatiation and link training have completed (if enabled) and that a a valid 64b/66b input data stream is being received by the high speed deserializer (HSRX port)

3. How to confirm where the issue is exactly? (T4240 Processor <---> TLK10232 <---> SFP+, i.e Through XAUI)

   A) I would suggest entering loopback mode in order to verify link. Please see attached document with explanation.

Loopback.pdf

Hi Malik Barton,

1. What will be the output clock at CLOCK_OUT PIN ?

    Since 0x1E.0x000D Bit [6]: CLKOUT_SEL[3:0] - 10x0 = Selects Ch B HS recovered byte clock as output clock, then the output clock frequecny should be the Ch B data rate divided by 4.

A) We have provided 156.25MHz input reference clock for Channel B. According to the details provided in the TLK10232 Datasheet, the output clock is calculated by REFCLKP/N (MHz) * SERDES PLL Multiplier. Therefore, 156.25 MHz * 4  = 625 MHz . When we probed the clock-out signal, it was not stable.

2. Which register need to be checked for Link UP?

Please read 0x03.0x0001 and 0x0008 for link status. Also see PCS_RX_LINK_STATUS in data manual.  Also PCS_RX_LINK_STATUS (Address 0x03, 0x0020, bit12) can be set to show link up on HS side. This bit indicates that auto-negotatiation and link training have completed (if enabled) and that a a valid 64b/66b input data stream is being received by the high speed deserializer (HSRX port)

A)   read 0x03.0x0001 - 0x0002, read 0x03.0x0008 - 0x8001 and read 0x03.0x0020 - 0x0004.

3. How to confirm where the issue is exactly? (T4240 Processor <---> TLK10232 <---> SFP+, i.e Through XAUI)

I would suggest entering loopback mode in order to verify link. Please see attached document with explanation.

A) In tlk10232_BringupProcedures_v2.pdf document, we have checked the 10G in 1:1 mode, HS / LS Test Patterns, with 156.25 MHz Refclk, Data Rate = 3.125Gbps

 mdio write 0x1E.0x0 0x8610
 mdio write 0x1E.0x1 0x0BA6
 mdio write 0x7.0x0 0x2000
 mdio write 0x1E.0x96 0x0
 mdio write 0x1E.0x8020 0x03FF
 mdio write 0x1e.0x0003 0x5848
 mdio write 0x1e.0x0004 0xD500
 mdio write 0x1E.0x000B 0x3F10
 mdio write 0x1E.0x000E 0x0008

 mdio read 0x1E.0x0010 - 0xFFFD (Error Counter)

we Read Back Serdes HS_AZ_DONE is not complete, HS_AGC_LOCKED is not locked, PLL Status is not locked

mdio read 0x1E.0x000F - 0x0 (Link Down)

4. I want to know about the Length Matching constraints,

4240 Processor <---> TLK10232 - 244.55 mm length

TLK10232 <---> SFP - 46.79 mm length

Regards,

Avinash N

 

Avinash N,

1. Is their any noise/jitter on the REFCLK signal? Could you try using a higher frequency REFCLK signal? What is the input data rate from the processor? What do you mean by "the clock-out signal, it was not stable"? Also the SERDES PLL Multiplier is ignored in when the TLK10232 is placed into KR mode. 

2. These reading confirm that the link is down, per the datasheet.

3.  Given these reading I suspect that there may be signal integrity issues. You should confirm the quality of your input signals by looking at the eye diagram with a high speed probe and scope. I highly recommend putting the device into loop-back mode if possible as it will help to confirm the link of the HS side of TLK10232 (TLK10232 <---> SFP).

4. By Length matching constraint are you referring to Intra-Pair Skew? If so the timing number are specified in the datasheet as tskew. The number change depending on the RX or TX. To convert these number to PCB trace length use the propagation delay of your PCB material. For FR4, propagation delay is estimated at ~6.535 ps/mm. Divide tskew by 6.535 to get the length that will meet the intra-pair skew limit. 

Hi Malik Barton,

1. What is the input data rate from the processor?

A) 3.125 Gbps for 1 Lane. 4 lanes used in our designed Board (XAUI connection).

2. What do you mean by "the clock-out signal, it was not stable"?

A) In TLK10232 Datasheet page No : 7, CLKOUTAP/N (Pin No :C9/C10), CLKOUTBP/N (Pin No :A9/A10). What Will be the clock out at c9,c10,a9 and a10 if we probed. Input reference clock is 156.25 MHz.

3. I highly recommend putting the device into loop-back mode if possible as it will help to confirm the link of the HS side of TLK10232 (TLK10232 <---> SFP).

A) How to check the device in loop back mode?

4. How to check whether the TLK10232 is working in a proper condition ?

Pins that are configured in Hardware are
MODE_SEL = 0,
ST = 0.

Regards,

Avinash N

Avinash N,

• Since 0x1E.0x000D Bit [6]: CLKOUT_SEL[3:0] - 10x0 = Selects Ch B HS recovered byte clock as output clock, then CLKOUTX pins should output clock frequency Ch B data rate divided by 4. If you are not seeing anything from the CLKOUTX pins try changing the value of CLKOUT_SEL[3:0] to the values below, resetting the data path each time. I believe your data fromt eh processor is coming into the LS not the HS, is this correct? Are these pins AC coupled?
  • 0110 = Selects Ch A LS recovered byte clock as output clock
  • 0111 = Selects Ch A LS transmit byte clock as output clock
  • 1110 = Selects Ch B LS recovered byte clock as output clock
  • 1111 = Selects Ch B LS transmit byte clock as output clock
• Please see the earlier file Loopback.pdf for details. The is a certain register setting required and data from the input on the HS side should be seen on the HS output.
• If I understand you correctly the Link UP and data error indicators mentioned earlier would be the best indicator. Are you looking for an additional indicator? 

Hi Malik Barton,

1. we have configured different options for CLKOUT_SEL[3:0]

A) But there was no output in CLOCKOUT PIN.

2. To Check the Loopback we have enabled the register LOOPBACK_TP_CONTROL,

Shallow local loopback mode

write 0x1E.0x0 0x8610

write 0x1E.0x001D 0x0000

write 0x07.0x0 0x20000

write 0x01.0x0096 0x0000

write 0x01.0x000B 0x0D11

write 0x1E.0x8020 0x03FF

write 0x1E.0x0004 0x9500

write 0x1E.0x0004 0xD500

write 0x1E.0xE 0x0008


mdio read 0x1E.0x000F - 0x0


Deep remote loopback mode

write 0x1E.0x0 0x8610

write 0x1E.0x001D 0x0000

write 0x07.0x0 0x20000

write 0x01.0x0096 0x0000

write 0x01.0x000B 0x0D18

write 0x1E.0x8020 0x03FF

write 0x1E.0x0004 0x9500

write 0x1E.0x0004 0xD500

write 0x1E.0xE 0x0008


mdio read 0x1E.0x000F - 0x0

A) Whether the Loopback procedure used for testing is correct ?


3. We have probed certain output pins. Whether with that can we able to debug further,

PRBS _PASS                  PRBS_PASS=0 indicates that a PRBS error is detected.
LS_OK_OUT_B               LS_OK_OUT_B=0: Channel B Transmit lanes not aligned.

and the LOS signal is behaving in different manner, Some times it is showing Signal Detected and Some time it is showing as Loss of signal.

Regards,

Avinash N

Avinash N,

To help me further understand you setup, I will ask some more questions, could you also post or private message me your schematic?

1. Here are some question is reference to CLKOUT issue: 

• Have you tried all combinations of the CLKOUT_SEL[3:0] register?
• When you are testing your system where is the data input, Channel A or B?
• Also just to confirm your are still not seeing the PLL lock at any time (please check HS_PLL_LOCK and  LS_PLL_LOCK ) ?
• Your processor is sending data into the LS side of the device and optical module is connected to the HS side correct?
• What encoding does the data have on the LS side? In the 10GBASE-KR mode, the TLK10232 performs serialization of the 8B/10B encoded XAUI data stream presented on its low speed (LS) side data inputs.
• What are the values of REFCLK_SW_SEL and LS_REFCLK_SEL registers? Also, REFCLKxP/N must be AC coupled, if this signal is not used this signal shouuld be pulled-down to GND through a shared 100Ohm.

2. The loop back procedure looks correct. Focus for now on Shallow local loopback mode. make sure to select the appropriate clock when debugging. Also after data path reset please make sure to wait 1000ms before continuing with testing/configuration, if you are not already. Values for the HS_SERDES_CONTROL_3 register may change in this mode.

3. These signals may not be representing the system correctly until a good link is established internally (during loop back) and externally. Here are some question is reference to these issues: 

• When probing PRBS_PASS was teh PRBS test mode enabled?
• LOS is tied directly to the peak to peak voltage of the input waveform?
• Does this or the common mode of your input signal change while testing?

Hi Malik Barton,

                              We Have Found an issue with respect to Schematic, TESTEN Pin is Pulled High. But as per TI recommendation it should be pulled Low. After TESTEN Pin is pulled Low, We can able to get the clock at CLOCKOUT Pin (156.25/4 = 39.05MHz).

But if we tried to access the register of the TLK10232 through MDC and MDIO it is Showing as 0xFFFF for all the register. (TESTEN - '0')

We can access the register of the TLK10232 through MDC and MDIO if TESTEN is pulled High. (TESTEN - '1').

What can be the issue?

What is the use of the TESTEN ? Whether it should be High or Low ?

Whether if we made TESTEN - '0' and Why we can't able to access the TLK10232 through MDC and MDIO ?

Regards,

Avinash N

Avinash,

TESTEN should be low for proper operation (TESTEN = 0). You should be able to use MDIO when TESTEN = 0. Please make sure you are following sections 7.6 and 7.7  of the TLK10232 datasheet. I suggest adding a 33bit of 1's for preamble in the MDIO communication, this may help with your issue. Also please ensure that you are following the MDIO frame format is as follows:

ST /OP / PHYADDR / DEVTYPE / TA / (ADDR/DATA)  16BITS

Hi Malik Barton,

There was some progress Today. We can able to access the TLK10232 Register through MDC and MDIO lines.

we have followed the bellow procedure,

1. Reset device (write a 1 to 0x1E.0000 bit 15 or assert RESET_N pin)
2. Make sure the reference clock selection (156.25 MHz or 312.5 MHz) is correct – this is done through register 0x1E.001D bit 12 (default is 156.25 MHz).
3. Disable auto-negotiation by writing 1’b0 to 0x07.0000 bit 12
4. Disable link training by writing 16’h0000 to 0x01.0096
5. Write 16’h03FF to 0x1E.8020. This allows the link settings that would normally be configured through KR training to be configured manually instead.
6. Depending on the link conditions, you may need to change the default configuration of 0x1E.0003 and 0x1E.0004. For optical connections, we typically recommend changing HS_ENTRACK (0x1E.0004 bit 15) to 1’b1 and HS_EQPRE (0x1E.0004 bits 14:12) to 3’b101. This can be a starting point, but you may need to do some BER testing to optimize the values.
7. Issue a data path reset by writing 1’b1 to 0x1E.000E bit 3.

mdio write 0x1E.0x0000 0x8610

mdio write 0x1E.0x0001 0x0BA6

mdio write 0x7.0x0000 0x2000

mdio write 0x01.0x0096 0x0

mdio write 0x1E.0x8020 0x03FFF

mdio write 0x1e.0x0004 0x9500

mdio write 0x1E.0x0005 0xD500

mdio write 0x1E.0x000E 0x0008

After that we performed read to check the Link Status,

mdio read 0x1E.0x000F - 0x5803

mdio read 0x07.0x0001 - 0x98

mdio read 0x03.0x0020 - 0x4

Which Indicates the Link Is Down.

1. What can be the issue ?

2. Whether any changes has to be done on the register configuration part, So that we can able to achieve LINK UP ?

3. Any Other procedure to check the Link is established ? (i.e, Processor <---> TLK10232(LS) and TLK10232 <---> SFP+ (HS))

Regards,

Avinash N

 

Avinash,

It seems you are having issues on the HS side of TLK10232. I would monitor 0x1E.0x0010 value and minimize the value recorded by changing the registers 0x1E.0x0004 and 0x1E.0x0005. The methods I have previously mentioned are the best ways to help determine the optimum settings for your link.

Hi Malik Barton,

1. There was a little bit progress. We have made certain changes in U-Boot. After that we can able to achieve as shown bellow.

mdio read 0x1E.0x000F - 0x5C03

mdio read 0x07.0x0001 - 0xbd

mdio read 0x03.0x0020 - 0x1005

But we cannot able to perform ping . We cannot able to perform any testing.

 

2. We are trying to perform Ping operation between Our customized Board and EVM of T4240 freescale processor.

But on the TLK10232 is showing issue. Register details for reference,

mdio read 0x1E.0x000F - 0x1803

mdio read 0x07.0x0001 - 0x88

mdio read 0x03.0x0020 - 0x4

Which Indicates the Link Is Down.

Question

1. How to test the First case?

2. What can be the issue in the second case? Whether the SFP+ and XFI cannot able to negotiate ?

3. whether we can able to find the throughput using internal Loopback ?

 

Regards,

Avinash N

Avinash ,

First Case:
Could you explain the block diagram and the test you are trying to run? What is transmitting into SFP+ 1 and 2 modules?

Second case:
Could you put TLK10232 into Shallow loopback mode and confirm the issue is not on the LS side. This is how i would begin to test the second case. If the SFP+ and XFI cannot able to negotiate it may be that the LS and HS lanes are out of alignment and this is causing issues.

When you say throughput are you referring to the output data of your system? If so I would measure the output waveform directly once in the desired configuration. Does this answer your question?

Hi Malik Barton,

                         1.  In First case, we are trying to make channel A as transmitter and channel B as receiver and we are performing a ping from channel A to channel B. Our TX and RX packet counter are getting incremented but there is no ARP response. we are syspecting whether there is proper communication is established between Channel A and B. Below is the configuration what we are doing for both the channel,

mdio write 0x1E.0x0000 0x8620; mdio write 0x1E.0x001D 0x000; mdio write 0x1E.0x0001 0x4B04;

mdio write 0x07.0x0000 0x2000; mdio write 0x01.0x0096 0x0000; mdio write 0x1E.0x000E 0x0008; mdio write 0x01.0x9000 0x024D;
mdio write 0x1E.0x8101 0x0004; mdio write 0x1E.0x8100 0x0004; mdio write 0x1E.0x8100 0x0000; mdio write 0x01.0x9001 0x0200
mdio write 0x07.0x0000 0x3000; mdio write 0x01.0x0096 0x0002; mdio write 0x01.0x9005 0x1C00; mdio write 0x1E.0x0002 0x831C;

mdio write 0x1E.0x0003 0xC8A8; mdio write 0x1E.0x0004 0xE500; mdio write 0x07.0x0000 0x3200

Channel A:

mdio read 0x1E.0x000F - 0x5d03; mdio read 0x07.0x0001 - 0xbd; mdio read 0x03.0x0020 - 0x1005

Channel B:

mdio read 0x1E.0x000F - 0x5503; mdio read 0x07.0x0001 - 0xbd; mdio read 0x03.0x0020 - 0x1005

                             2. As per your suggestion we made the TLK10232 into shallow loopback mode and we read the status as shown below,

mdio write 0x1E.0x0000 0x8620;mdio write 0x1E.0x001D 0x0000; mdio write 0x1E.0x0001 0x4B04; mdio write 0x07.0x0000 0x2000; mdio write 0x01.0x0096 0x0000;

mdio write 0x1E.0x000B 0x0D18; mdio write 0x1E.0x8020 0x03FF; mdio write 0x1E.0x0003 0xC8A8; mdio write 0x1E.0x0004 0xE500; mdio write 0x1E.0x000E 0x0008;              

Channel A:

mdio read 0x1E.0x000F - 0x5003; mdio read 0x07.0x0001 - 0x88; mdio read 0x03.0x0020 - 0x4

Channel B:

mdio read 0x1E.0x000F - 0x1803; mdio read 0x07.0x0001 - 0x88; mdio read 0x03.0x0020 - 0x7

Question:

1. Any Suggestion or Idea to test First condition.

2. On Second case one channel is getting LINKED UP and another channel is not and showing High BER condition detected. What can be the issue ?

Regards,

Avinash N

Avinash,

Sorry for the late reply.

1. It seems that you still have Auto-negotiation and Link Training enabled on. (See LT_TRAINING_CONTROL, AN_ENABLE). Are you trying to manually trigger the loading of TX default values? I would set DEFAULT_TX_TRIGGER_EN, DEFAULT_TX_LOAD_TRIGGER and AP_SEARCH_MODE[2:0] to their default values. Do not write to 0x01.0x9000 and 0x9005, leave at default value. Please retest.

2. It seems that you still have Auto-negotiation and Link Training enabled on. (See LT_TRAINING_CONTROL, AN_ENABLE) Please turn this off and retest.

Also what encoding are you using on the input, is it 8b/10b?

Hello,
May I know what was the flow used to generate uboot? is it petalinux? did you use the default uboot image from petalinux or custom one?

Thanks