AM6412: AM64X Ethernet Backup Boot Mode

Hello Ragesh,

Thanks for the query !

Please allow me some time to internally check.

I will try to get back to you with a response early next week.

Your patience is appreciated !

Best Regards

Anastas Yordanov

Are there any progress on this topic? 

Regards,

Samuele

Hello Samuele, Ragesh,

Sorry for the time gap in my response.

I try to interpret the below of your notices:

Ragesh Nair said:

Primary boot mode Ethernet RGMII, Link params programmed by SW, BOOTMODE[9..3] = 0b0010100
Result: BOOTP/DHCP packets are captured by tcpdump --> OK

Ragesh Nair said:

Primary boot mode Ethernet RGMII, MDIO PHY scan, BOOTMODE[9..3] = 0b0000100
Result: No activity on the Ethernet interface and MDIO --> Not OK

and assume that the problem has something to do with the AM6412 MDIO  scanning involved in the second case (Link info bit7 = 0).

 In the AM243x/AM64x Silicon Errata 1.0/2.0 there is advisory note - i2329 regarding a AM243x MDIO hardware state machine issue, applicable to both 1.0 and 2.0 silicon revisions: 

 It reads:

The workaround looks like:

To fix this behavior, it is recommended to use the appropriate MDIO Manual mode driver from the SDK. 

I hope this helps.

Looking froward to your feedback !

Thanks

Best Regards,

Anastas Yordanov

After multiple readings of the Technical Reference Manual (Rev. I), I think there are some inconsistencies across the manual and the SK development board as well:

"Table 4-51. Ethernet Boot Parameter Table" states (offset 259) that the interface expected could be (based on the parsed pins configuration):
- 0 = RGMII with internal delay
- 1 = RGMII with external delay
- 2 = RMII

Though checking the boot mode pins ("Table 4-4. Pirmary Boot Mode Selection", "Table 4-5. Backup Mode Selection", "Table 4-20. Ethernet RGMII Boot Configuration Fields", "Table 4-23. Ethernet Backup Boot Configuration Field") the only phy-mode configuration available for RGMII is "RGMII with internal Tx delay".

Moreover looking at the schematic of the SK evaluation board featuring a "dp83867" as well, it can be observed that the PHY attached to the RGMII2 interface features strap resistors configuring:

- PHY address to 0x1 (I believe this should be 0x0 to let the ROM code find the correct PHY)

- RX internal delay PHY mode (in conflict with what expected by the ROM code)

I might have missed something here, could please help?

Best regards,

Samuele

Hello Samuele,

Samuele Favazza said:

"Table 4-51. Ethernet Boot Parameter Table" states (offset 259) that the interface expected could be (based on the parsed pins configuration):

ROM can switch over internal and external Tx delay as part of it's functionality, however external TX delay option is not supported  in AM64x devices. That's why the other option is reserved:

Samuele Favazza said:

- PHY address to 0x1 (I believe this should be 0x0 to let the ROM code find the correct PHY)

PHY address is not needed. ROM performs PHY scan on MDIO0. 

These are the pins configured and used by ROM:

This is the initialization steps performed by ROM for Ethernet boot:

Regards,

Stan

Hi Anastas,

thanks for the detailed reply. Indeed, there seems to be a problem with the MDIO scanning. I probed the MDIO lines and was not able to see any activity there. 

Unfortunately, the workaround does not apply to this issue, as we are still tin the ROM code domain, which I can't control.

Best regards,

Ragesh

Hi Stan,

just to be sure: For primary Ethernet boot I can choose between with MDIO scan (Link info bit = 0) and without MDIO scan (Link info bit = 1). Which option is used for the backup Ethernet boot? With or without MDIO scan? 

Regards,

Ragesh

Hi Ragesh,

I will need to check internally. This is not described in TRM unfortunately.

Regards,

Stan

Hi Stan,


Thank you for your answer. Just to ensure I understood the concept of RGMII internal delays: an RGMII with "internal TX delay" means that the "TX clock skew" is provided by the PHY, e.g. by the "dp83867", right?

My assumption is based on what I observed playing around with the "phy-mode" configuration in our kernel device tree:
- "rgmii-id" (internal delay on both TX/RX paths): results in the dp83867 RGMIIDCTL (0x86) register to be configured to 0x0077
- "rgmii-rxid" (internal RX delay only): results in reg 0x86 set to 0x0007
- "rgmii-txid" (internal TX delay only): does not work (I cannot understand why)

Is my understanding of "internal delay" correct?

Also, has anybody successfully managed to use the backup Ethernet boot on an SK evaluation board? (we have a SKEVM SK-AM64 PROC100E3).

Best regards,
Samuele

Hi  Samuele, Ragesh,

Thanks for the questions !

Stan is out of office today and he will be able to respond after Tuesday (March-17). I think I can comment on some of the questions.

Ragesh Nair said:

just to be sure: For primary Ethernet boot I can choose between with MDIO scan (Link info bit = 0) and without MDIO scan (Link info bit = 1). Which option is used for the backup Ethernet boot? With or without MDIO scan? 

An internal query has been made. We are awaiting the response from our AM64x ROM Ethernet boot expert.

Samuele Favazza said:

- RX internal delay PHY mode (in conflict with what expected by the ROM code)

I see this as a conflict too, as the AM6412 CPSW MAC RGMII transmitter already applies "TX internal delay" for the ROM booting over Ethernet.

Please try configuring the corresponding strap to disable the DP83867 PHY RX delay, as no additional delay needed on this path.

Samuele Favazza said:

Thank you for your answer. Just to ensure I understood the concept of RGMII internal delays: an RGMII with "internal TX delay" means that the "TX clock skew" is provided by the PHY, e.g. by the "dp83867", right?

Samuele,

As per my understanding when TX internal delay is discussed in context of ROM Ethernet booting, the AM6412 CPSW RGMII TX side and NOT the  PHY's TX delay is referred. The PHY RX should be actually the "external side". The delay is applied by default at AM6412 TX, and no additional delay shall be inserted on the dp83867 PHY receiver side (which will be external delay). This means, the "internal TX delay" between clock and data is inserted by the AM6412 CPSW RGMII TX. 

Looking forward to your feedback !

Thanks

Best Regards

Anastas Yordanov

Anastas Yordanov said:

I see this as a conflict too, as the AM6412 CPSW MAC RGMII transmitter already applies "TX internal delay" for the ROM booting over Ethernet.

Please try configuring the corresponding strap to disable the DP83867 PHY RX delay, as no additional delay needed on this path.

Hello Anastas,

Thanks for the prompt reply. Before messing around with the PHY strap resistors I performed few tests to evaluate how the "DP83867" behaves when modifying its internal delays. This is my setup:
– powerup the board and reach the Linux userspace shell
– use an "mdio-tools" utility to interact with the PHY register bank
– in particular the "RGMIIDCTL" (0x86) register has been modified
– a simple "ping" call proves whether the SoC is still capable to communicate with the PHY

It turns out that any PHY TX internal delays are supported (0.25ns till 4.00ns, i.e. 0x0 till 0xf reg value range). On the other hand the PHY RX internal delay MUST be kept within the [1.00, 3.50] ns interval (i.e. 0x3 till 0xd reg value range). Based on this observation it seems that the strap resistors on the SK board are correctly configured.

To further test the SoC PHY interaction, could you tell me which registers is the ROM code accessing to configure the RGMII TX internal delay?

More in general, would it be possible to inspect the ROM source code? It would greatly help us understand what is going on during the backup Ethernet boot process.

Best regards,
Samuele

Hello Samuele,

Let's consider the following AM64x TRM section as a reference for RX path and TX path direction:

Samuele Favazza said:

To further test the SoC PHY interaction, could you tell me which registers is the ROM code accessing to configure the RGMII TX internal delay?

As far as I know a RGMII TX internal delay (ID) of minimum 1.2 ns - see Section, CPSW3G RGMII Timing of the AM6412 Datasheet is automatically provided by default  from the AM6412 CPSW RGMII transmitter hardware upon POR. The ROM code does not need to touch any AM6412 register to enable such delay in software.

Regarding the CPSW RGMII Data Receive (i.e. PHY to CPSW) path 

Based on these AM64x Datasheet TX path Output switching / RX path Timing requirements following PHY 0x0086 RGMIIDCTL register values shall be tried.

In my understanding the mentioned configuration: 

- "rgmii-rxid" (internal RX delay only): results in reg 0x86 set to     >= 0x0003 - 0x000f

Samuele Favazza said:

On the other hand the PHY RX internal delay MUST be kept within the [1.00, 3.50] ns interval (i.e. 0x3 till 0xd reg value range).

Q: How did you come to this conclusion. Why 0xE and 0xF values are not acceptable ?

I am still awaiting the requested ROM Ethernet boot owner's confirmation.

Thanks

Best Regards

Anastas Yordanov  

Hello Anastas,

I really appreciate the clarification, I totally missed the timing characteristic of the AM6412 SoC.

Anastas Yordanov said:

Q: How did you come to this conclusion. Why 0xE and 0xF values are not acceptable ?

To answer your questions let me share with you few screenshots from 2 of our devices based on the AM6412 SoC (it is assumed the TX delays is always left at 0x0):

 which confirms your statements about the range being 0x3..0xF.

On the second device (the one where we would like to adopt the Ethernet backup booting mode) I observed a slightly different behavior, which led me to consider the range 0x3..0xd as safe:

To prove whether a value works or not I just executed a ping request. Hope answers your question.

In any case, based on your explanation and the documentation screenshot you shared, it is clear that the SK EVM board has the strap resistors correctly configured. Yet is not clear why no activity is detected when falling back to the Ethernet backup boot mode.

Best regards,
Samuele

Hello Samuele,

Yes I understand that there might be some variations in delay compensation values (because of difference in device sample to sample, PCB traces characteristic impedance, environment conditions) needed on the RX path between AM6412 and the PHY.  

I reached out to our AM64x Ethernet expert.

Samuele Favazza said:

Also, has anybody successfully managed to use the backup Ethernet boot on an SK evaluation board? (we have a SKEVM SK-AM64 PROC100E3).

I would like to once again bring your attention to this earlier notice from myside in the thread (see highlighted in orange):

The Ethernet expert also highlighted a second ROM Ethernet boot- related advisory note i2331 from the AM64x/AM243x Silicon Errata 1.0/2.0 

The Ethernet expert advices to refrain from using the ethernet bootmode configured by ROM.

Regarding the ROM behavior on the CPSW delay configuration - the expert statement is: 

"When the CPSW external delay is configured, it is handled by the PHY itself, but I cannot confirm if the ROM will setup the delays or not."

We have to wait on the ROM code team experts to respond to your questions regarding:

Ragesh Nair said:

For primary Ethernet boot I can choose between with MDIO scan (Link info bit = 0) and without MDIO scan (Link info bit = 1). Which option is used for the backup Ethernet boot? With or without MDIO scan? 

Samuele Favazza said:

Yet is not clear why no activity is detected when falling back to the Ethernet backup boot mode.

Thanks

Best Regards

Anastas Yordanov

Hello Anastas,

It seems clear that the silicon rev. 2 & 1 do not offer a reliable Ethernet backup boot mode. Is there a plan to correct this issue in the next silicon revision (is it at all planned another silicon revision)?

Best regards,
Samuele

Hello Samuele,

Let me check with the AM64x team and get back to you !

Please expect that some delay is possible.

Thanks

Best Regards

Anastas Yordanov

Hello Samuele,

I would like to apologize for the two week gap in my response !

I hope we could  have the answer of the team by the beginning of the next week.

Thanks for your understanding and patience !

Best Regards

Anastas Yordanov

Hello Anastas,

Thank you for keeping me informed. Patiently waiting.

Regards,

Samuele Favazza

Sorry, there is no future silicon revision planned for AM64x.  Ethernet can be used on AM64x in normal operation, but not reliably as a boot source

Regards,

James