C6474, NDK, Connect Speed

Alex

Other Parts Discussed in Thread: TMS320C6474

So I have updated C6474 EVM's UDP echo client sample application to run on our custom platform. (We are using a Broadcom PHY, not a Marvell switch like on the EVM.)

We are operating over copper in SGMII mode, and the issue I have is between the DSP and the PHY. (The PHY and the outside world have no problems connecting at any speed, be it 100 Mbps or 1 Gbps.)

Here's the rub. No matter what configuration bits I set, the DSP will only talk to the PHY in 1 Gbps mode, even if I explicitly try to force 100 Mbps. In addition, it will only sync if I enable auto-negotiation (in the DSP's SGMII register). If I disable auto-negotiation, regardless of what speed I try to force, I can't get a sync, i.e., it receives no packets from the PHY. Now, with auto-negotiation having to be enabled, my solution to "force" settings is to also update the capability advertising settings. But even if I tell it to only advertise 100 Mbps, it still connects at 1 Gbps. (Note: for good measure, I explicitly set the speed to 100 Mbps, but according to the PHY's datasheet, those settings, I think, are ignored if auto-negotiation is enabled. I could be wrong.)

Probably for the same reasons, regardless of what settings I use, I cannot connect to any 100 Mbps devices... period. It's 1 Gbps or nothing.

And by "settings,' over the last three weeks, I am talking probably every possible combination of bits among the PHY, EMAC, MDIO, and SGMII register settings. With that in mind, I also realize it is probably impossible for anyone to blindly know what is wrong on my system. So generically, and from the DSP's perspective only (I have a good handle on all the PHY's registers I think), what changes need to be made to switch between 1 Gbps operation to 100 Mbps operation?

Asked another way, Section 2.1.2 of the EMAC-MDIO (spruag08a) states that "[t]he SGMII protocol also allows for dynamic switching between 10/100/1000 Mbps modes." How do I set it up to do that?

over 14 years ago

0 Bill Taboada over 14 years ago

TI__Mastermind 42545 points

I want to make sure I understand where you are seeing the problem. It appears from your description that the auto-negotiation on the copper side is operating correctly and that the PHY is properly detecting the speed of it's link partner. The SGMII auto-negotiation on the serdes side operates a little differently. The PHY will detect the rate that is needed on the copper interface and advertise that rate to the DSP. The DSP will acknowledge that rate. It can't change the rate but only acknowledge that it's received the negotiation message. Only after this procedure has been completed the DSP will start to transmit data to the PHY. Auto-negotiation is required by the DSP and it will not transmit or receive packets until it has received the negotiation information from the PHY. Note that regardless of the negotiated rate the SGMII serdes interface will always operate at 125MHz as per the SGMII specification. If a lower speed is needed due to the connection on the copper interface, the PHY and MAC will simply repeat the data multiple times to accomodate the lower speed. Therefore if the PHY is connected to a 100Mbit device on the copper side of the interface it will go through the following steps.

- The PHY will complete the auto-negotiation with the link partner over the copper interface and determine that it's communicating with a 100Mbit device.

- The PHY will send an SGMII auto-negotiation command to DSP defining the interface speed as 100Mbit. The DSP will acknowledge that speed.

- The PHY will begin to repeat each data word ten times on the SGMII interface to compensate for the slower speed on the copper interface. The DSP will do the same with the transmit data.

If the interface is defined as 10Mbit it will repeat each data word one hundred times. In order to accomidate for the different rates on the copper side SGMII auto-negotiation is alway needed. The only instantance that auto-negotiation on the SGMII side wouldn't be needed is if it was connected to something like a switch where the link was always 1Gbit regardless and the other rates were accomodated on the other end of the switch fabric. The concept of forcing an SGMII link is usually used when attempting a MAC-to-MAC connection where neither party can intitialize the SGMII auto-negotiation. As stated previously the DSP doesn't support this mode so SGMII auto-negotiation is always required.

0 Alex over 14 years ago in reply to Bill Taboada

Expert 2430 points

That makes sense about the DSP <--> PHY SGMII auto-negotiation. The "helloWorld" sample I have running on Spectrum Digital's C6474 EVM has auto-negotiation disabled, but as you pointed out at the end, the EVM uses a switch. My custom board uses a simple PHY. The sample app's comment doesn't match the implementation, so I kept overlooking it (setting was to disable auto-negotiation, but the comment was "/* enable auto-negotiation */" which turned out to be my initial roadblock).

So, now, though, as I stated in my original post, I can only connect at 1 Gbps speeds. If I connect my board to anything not capable of 1 Gbps speeds, I receive no packets. (Windows is always establishes a connection, and dumping out the PHY registers prove to me that it is able to communicate to the switch, i.e., no errors, and the "partner's advertised capabilities" are always correct.) But on the DSP, I receive absolutely no Rx (incoming) interrupts, that is, unless I am connected to a 1 Gbps device.

So presuming I have auto-negotiation enabled across the entire path, any clues which specific control registers I should looking at on the DSP side?

0 Bill Taboada over 14 years ago in reply to Alex

TI__Mastermind 42545 points

Chapter 4 of the TMS320C6474 EMAC/MDIO Users Guide (SPRUG08A) lists the registers for the SGMII interface. Let's start with those. As detailed in section 2.18.4 of that document you should be configuring as follows.

The following setup shows the CPSGMII setup for this mode of operation:

1. Setup the CPSGMII and enable auto-negotiation:

MR_ADV_ABILITY = 0x1 /* MAC to PHY config register */
CONTROL = 0x1 /* Enable auto-negotiation, slave mode */

2. Poll the STATUS register to determine when auto-negotiation is complete without error. The
AN_ERROR bit in the STATUS register will be set if the mode was commanded to be half-duplex
gigabit.

The status register is described in section 4.5. You should be able to check that the negotiation has completed and that the link is up.

0 Alex over 14 years ago in reply to Bill Taboada

Expert 2430 points

Thanks! That was problem. I had it set to be the master, not the slave. Setting it to operate in slave mode, I can now connect to the outside world at both 100 Mbps and 1 Gbps. I am somewhat surprised that this works, though, because our board's PHY is bootstrapped to come up in slave mode, hence why I was telling the DSP to operate in master mode.

Interestingly, though, setting it to slave mode "auto-magically" changes the MR_ADV_ABILITY register to 0x1; when set to master, it reads 0x9801 which reflects what I am setting it to. I guess I need to go figure that one out.

Back to the master/slave setting: is there a general recommendation? Is it better for the DSP to be the master or the PHY? (Again, theoretically, I can configure the PHY either way.) Or does it not really matter?

0 Bill Taboada over 14 years ago in reply to Alex

TI__Mastermind 42545 points

If you look at the SGMII specification you will see that it doesn't discuss the connection in terms of master and slave. It discusses the connection in terms of MAC and PHY where the PHY initiates the negotiation. I think the that the concept of master and slave appeared to describe MAC to MAC connections and define which MAC would initiate the negotiation. If you think of the master device as the device that initiates then the normal expected configuration is that the PHY is master and the MAC is slave. This makes sense since the speed of the interface is determined by the connection at the copper end of the PHY. Matching the SGMII transfer rate to the speed of the copper interface eliminates the need for any significant buffering in the PHY.

Concerning the MR_ADV_ABILITY register, in the EMAC users guide it states that it will correspond to the tx_config_reg[15:0] register value in the SGMII specification. Basically the SGMII spec states that the PHY will inform the MAC of it's present operating condition using the tx_config_reg based on the table below.

Bit # tx_config_reg sent from PHY to MAC tx_config_reg sent from MAC to PHY

15 Link: 1-link up; 0-link down 0: reserved

14 Reserved for auto-neg ack 1

13 0: Reserved for future use 0: Reserved for future use

12 duplex 1:full; 0:half 0: Reserved for future use

11:10 speed: 11:res;10:1G;01:100M;00:10M 00: Reserved for future use

9:1 000000000: Reserved for future use 000000000: Reserved for future use

0 1 1

So the 0x9801 that your seeing in the MR_ADV_ABILITY register translates into a message from the PHY that says the link is up, the link is full duplex and the link is 1000BASE-T. Does the value change if you connect to the PHY to a 100M device?

0 Alex over 14 years ago in reply to Bill Taboada

Expert 2430 points

So as a preface, as I mentioned, I am just currently running the NDK's "helloWorld" demo application. (Actually, it is an updated version made by one of your colleagues to run on Spectrum Digital's C6474 EVM.) I, in turn, have been updating that to 1.) run my own board initialization stuff (PLL, DDR, etc.) instead of the EVM's, and 2.) configure my Broadcom PHY instead of the EVM's Marvell switch.

So, the MR_ADV_ABILITY register is initialized in the sgmiiInit() function at start-up. It is currently initialized to 0x9801 (full duplex, 1 Gb).

Now with that out of the way, to answer your question, when set the DSP to operate in slave mode, the register auto-magically updates itself when connecting to either a 1 Gb switch or a 100 Mb switch. I keep using the term "auto-magically " because when I put a watchpoint on it, it is happening inside the NDK kernel. (Since I don't have the source code for that, I can't be any more specific since I only see the disassembly and no symbols) Below is a summary of my experiments:

Mode	CONTROL	MR_ADV_ABILITY	MR_ADV_ABILITY Init. Value	Results
DSP slave (1 Gb)	0x00000001	0x00000001	0x00009801	able to echo packets between PC and DSP
DSP slave (100 Mb)	0x00000001	0x00000001	0x00009801	able to echo packets between PC and DSP
DSP master (1 Gb)	0x00000021	0x00009801	0x00009801	able to echo packets between PC and DSP
DSP master (100 Mb)	0x00000021	0x00009801	0x00009801	NOT able to echo packets between PC and DSP
DSP master (100 Mb)	0x00000021	0x00009801	0x00009401	NOT able to echo packets between PC and DSP

What is perplexing is my PHY is configured as the master (that's how it's boot-strapped to come up, and dumping its status register confirms this), yet somehow when the DSP is configured to be the master, a link is still established and I am able to echo packets back and forth. As a footnote, I have software control (in the PHY) to configure it as master or slave, but only at 1 Gb. (Only its 1000BASE-T control register has the setting; the 100BASE-TX/10BASE-T modes seem restricted to the chip's input pin at reset, i.e., boot-strapped.)

So I don't know if I have question in all this. In short, the DSP software works when the DSP is set to be the slave, so I should probably just count my blessings and move on.

0 Bill Taboada over 14 years ago in reply to Alex

TI__Mastermind 42545 points

I must have misinterpreted the documentation. If possible could you repeat the above experiment and dump the MR_LP_ADV_ABILITY and the STATUS registers as well? I belived that the function of the MR_ADV_ABILITY register changed depending on whether the SGMII was set up as slave or master but I must have been wrong. The MR_LP_ADV_ABILITY register is read-only and should reflect what the link partner is sending. Hopefull that value will vary with the speed at the copper side of the interface as I originally hoped MR_ADV_ABILITY would. I suspect that the 1Gb master mode works because both sides are configuring for 1Gb, the DSP side because we're commanding it and the PHY side due to auto-negotiation. I'm not sure how that effects the negotiation but I'm hoping the STATUS register dump will give us a clue. Regardless both sides are configured at the same rate so traffic can flow.

0 Alex over 14 years ago in reply to Bill Taboada

Expert 2430 points

100 Mb Switch : MR_LP_ADV_ABILITY reads 0xD401

1 Gb Switch: MR_LP_ADV_ABILITY reads 0xD801

spruag08a is not specific on this. Where can I find documentation to decode this, or to quote spruag08a, where can I get a copy of the SGMII specification to see what tx_config_reg [15:0] is defined as?

To muddy the waters, though, is this value (read on the DSP) referring to the PHY as the partner, or is it a what the PHY thinks its partner's abilities are? To elaborate, whether no matter how I configure the PHY (i.e., force it to 100 Mb or 1 Gb), it always reports the same value unless I connect to a different switch. Now, the PHY's MR_LP_ADV_ABILITY register (equivalent) does change and report the correct values accurately describing the switch I am connected to, so i am just wondering if that is what is being passed up to the DSP.

0 Bill Taboada over 14 years ago in reply to Alex

TI__Mastermind 42545 points

SGMII is actually a Cisco specification. If you do a google search on Cisco SGMII specification you should get a PDF copy of the spec.One of the better descriptions of the SGMII auto-negotiation can be found in chapter 10 of the Xilinx document at the following link. http://www.xilinx.com/ipcenter/1gbsx_phy_lounge/gig_eth_pcs_pma_ug155.pdf It shows the flow of information from the copper side auto-negotiation and the SGMII auto-negotiation. The paragraph describing SGMII auto-negotiation in the SGMII spec is below. I copied table 1 defining tx_config_reg[15:0] from that spec into the earlier post above.

Control information, as specified in Table 1, is transferred from the PHY to the MAC to signal the change of the control information. This is achieved by using the Auto-Negotiation functionality defined in Clause 37 of the IEEE Specification 802.3z. Instead of the ability advertisement, the PHY sends the control information via its tx_config_Reg[15:0] as specified in Table 1 whenever the control information changes. Upon receiving control information, the MAC acknowledges the update of the control information by asserting bit 14 of its tx_config_reg{15:0] as specified in Table 1.

0 Bill Taboada over 14 years ago in reply to Bill Taboada

TI__Mastermind 42545 points

Did the above exchange answer your questions? If so could you do me a favor and click the Verify Answer button on this post? If not let me know what other details I can provide and I'll try and dig up the information.

0 tugrul mutlu over 14 years ago in reply to Bill Taboada

Intellectual 365 points

Hi Alex,

I also use a Broadcom Chip (5461) as PHY. But for now i couldnt establish a link between PHY and SGMII.

I configure PHY and set its mode for using its SERDES interface(GBIC , Because 6474 SGMII interface also use serdes ). I also try lots of combination in order to establish the link between PHY and SGMII. Although i have a 1 gb copper link, i couldn do it.

This is my sgmii configuration function. After calling this function status register is always 0x30 means link is down.

init_sgmii()

{

SGMII_Config SgmiiCfg;

SgmiiCfg.masterEn = 0x0;

SgmiiCfg.loopbackEn = 0x0;

SgmiiCfg.txConfig = 0x00000e13;

SgmiiCfg.rxConfig = 0x00081013;

SgmiiCfg.auxConfig = 0x0000000b;

SgmiiCfg.modeOfOperation = 0x1; // Autoneg

SGMII_reset();

SGMII_config (&SgmiiCfg);

}

what could be the problem, do you have any idea ?

0 Alex over 14 years ago in reply to tugrul mutlu

Expert 2430 points

tugrul,

Did you base your application on one of the sample/demo apps? I ask because my init_sgmii() function was one of the things that stump me, too, for a day or two. The emac_open() call, which is in the emac_drv.c module, overwrites the SgmiiCfg settings. (This is based on the "helloWorld" demo.)

Make sure something similar isn't going on in your application.

The txConfig settings are somewhat hardware dependent. I measured our swing voltage with a scope--admittedly, not very successfully, but still...--and I am currently using the following settings:

#define RX_CFG_EQ(x)        (((unsigned int) (x))<<19)
#define RX_CFG_CDR(x)       (((unsigned int) (x))<<16)
#define RX_CFG_LOS(x)       (((unsigned int) (x))<<14)
#define RX_CFG_ALIGN(x)     (((unsigned int) (x))<<12)
#define RX_CFG_TERM(x)      (((unsigned int) (x))<<8)
#define RX_CFG_INVPAIR(x)   (((unsigned int) (x))<<7)
#define RX_CFG_RATE(x)      (((unsigned int) (x))<<5)
#define RX_CFG_BUSWIDTH(x) (((unsigned int) (x))<<2)
#define RX_CFG_ENRX(x)      (((unsigned int) (x))<<0)

#define TX_CFG_ENFTP(x)     (((unsigned int) (x))<<16)
#define TX_CFG_DE(x)        (((unsigned int) (x))<<12)
#define TX_CFG_SWING(x)     (((unsigned int) (x))<<9)
#define TX_CFG_CM(x)        (((unsigned int) (x))<<8)
#define TX_CFG_INVPAIR(x)   (((unsigned int) (x))<<7)
#define TX_CFG_RATE(x)      (((unsigned int) (x))<<5)
#define TX_CFG_BUSWIDTH(x) (((unsigned int) (x))<<2)
#define TX_CFG_ENTX(x)      (((unsigned int) (x))<<0)

SgmiiCfg.masterEn   = 0x0;              // Enable SGMII Slave
SgmiiCfg.loopbackEn = 0x0;              // Disable SGMII loopback

// DE=0,SWING=6,CM=1 = 800 mV swing @ Broadcom PHY inputs
SgmiiCfg.txConfig   = TX_CFG_ENFTP(1)       |
                      TX_CFG_DE(0)          |
                      TX_CFG_SWING(6)       |
                      TX_CFG_CM(1)          |
                      TX_CFG_INVPAIR(0)     |
                      TX_CFG_RATE(1)        |
                      TX_CFG_BUSWIDTH(0)    |
                      TX_CFG_ENTX(1);

SgmiiCfg.rxConfig   = RX_CFG_EQ(1)          |
                      RX_CFG_CDR(0)         |
                      RX_CFG_LOS(0)         |
                      RX_CFG_ALIGN(1)       |
                      RX_CFG_TERM(1)        |
                      RX_CFG_INVPAIR(0)     |
                      RX_CFG_RATE(1)        |
                      RX_CFG_BUSWIDTH(0)    |
                      RX_CFG_ENRX(1);

// Setup for 125 MHz ref clock
SgmiiCfg.auxConfig       = SGMIICFG_AUX_CONFIG;             // Enable PLL
SgmiiCfg.modeOfOperation = SGMII_MODE_OF_OPERATION_WITH_AN; // Auto Negotiation on

Which yields the following final settings:

SgmiiCfg
    masterEn        0x00
    loopbackEn      0x00
    modeOfOperation 0x01
    txConfig        0x00010D21
    rxConfig        0x00081121
    auxConfig       0x0000000B

But I can't promise that will work for you as it is also dependent on the PHY settings. If the above doesn't work for you, I'll post a dump of all my PHY registers. As you probably know by now, it's a fairly extensive list, but I already am dumping them out, so it would just be a matter of copying and pasting.

0 Alex over 14 years ago in reply to Alex

Expert 2430 points

In fact, here's the debug output my modified "helloWorld" demo app. Keep in mind that our H/W guy did a top-notch job of bootstrapping the PHY to come up exactly as needed, i.e., I literally have no need to initialize any of the registers. All I do is reset it, and that's only for good measure. That said, ignore the LED register settings as he customized their functionality in our hardware and they wouldn't apply to your platform undoubtedly.

Again, to be very clear, the register values below are simply a combination of the part's default values and how the PHY is bootstrapped to come up. That said, though, with these settings, and with my SgmiiCfg settings above, I am able to connect to the outside world at both 100 Mbps and 1 Gbps speeds. (I haven't bothered to test 10 Mbps speeds.)

Oh, and for the 1Ch shadow register values, I masked out the shadow register value for readability's sake, so don't let that throw you off.

[C64PLUS_3A] TCP/IP Stack 'Hello World!' Application
[C64PLUS_3A]
[C64PLUS_3A] emac_init: core 0, port 0, total number of channels/MAC addresses: 1/1
[C64PLUS_3A] MAC addresses configured for channel 0:
[C64PLUS_3A] xx-xx-xx-xx-xx-xx
[C64PLUS_3A] SGMII reset successful........
[C64PLUS_3A] SGMII config successful........
[C64PLUS_3A] emac_open core 0 port 0 successfully
[C64PLUS_3A] Registration of the EMAC Successful, waiting for link up ..
[C64PLUS_3A] Service Status: DHCPC    : Enabled :          : 000
[C64PLUS_3A] Service Status: DHCPC    : Enabled : Running : 000
[C64PLUS_3A] Port 0 Link Status: 1000Mb/s Full Duplex on PHY 1
[C64PLUS_3A] ----------------------------------------
[C64PLUS_3A] PHY Register Dump
[C64PLUS_3A] ----------------------------------------
[C64PLUS_3A] Register 00h        : 0x1000 (MII Control)
[C64PLUS_3A] Register 01h        : 0x796D (MII Status)
[C64PLUS_3A] Register 02h        : 0x0020 (PHY Identifier [MSB])
[C64PLUS_3A] Register 03h        : 0x60C1 (PHY Identifier [LSB])
[C64PLUS_3A] Register 04h        : 0x01E1 (Auto-Negotiation Advertisement)
[C64PLUS_3A] Register 05h        : 0xCDE1 (Auto-Negotiation Link Partner Ability)
[C64PLUS_3A] Register 06h        : 0x000F (Auto-Negotiation Expansion)
[C64PLUS_3A] Register 07h        : 0x2001 (Next Page Transmit)
[C64PLUS_3A] Register 08h        : 0x4215 (Link Partner Received Next Page)
[C64PLUS_3A] Register 09h        : 0x0200 (1000BASE-T Control)
[C64PLUS_3A] Register 0Ah        : 0x3C00 (1000BASE-T Status)
[C64PLUS_3A] Register 0Fh        : 0x3000 (IEEE Extended Status)
[C64PLUS_3A] Register 10h        : 0x0000 (PHY Extended Control)
[C64PLUS_3A] Register 11h        : 0x0301 (PHY Extended Status)
[C64PLUS_3A] Register 12h        : 0x0000 (Receive Error Counter)
[C64PLUS_3A] Register 13h        : 0x0000 (False Carrier Sense Counter)
[C64PLUS_3A] Register 14h        : 0x0000 (Receiver NOT_OK Counter)
[C64PLUS_3A] Register 15h[00h]   : 0x0000 (Receive/Transmit Packet Counter)
[C64PLUS_3A] Register 15h[01h]   : 0x0060 (Expansion Interrupt Status)
[C64PLUS_3A] Register 17h        : 0x0000 (Expansion Register Access)
[C64PLUS_3A] Register 18h[000]   : 0x0400 (Auxiliary Control)
[C64PLUS_3A] Register 18h[001]   : 0x0000 (10BASE-T)
[C64PLUS_3A] Register 18h[010]   : 0x0280 (Power/MII Control)
[C64PLUS_3A] Register 18h[100]   : 0x0000 (Misc Test)
[C64PLUS_3A] Register 18h[111]   : 0x7060 (Misc Control)
[C64PLUS_3A] Register 19h        : 0xFF1C (Auxilary Status Summary)
[C64PLUS_3A] Register 1Ah        : 0x043E (Interrupt Status Register)
[C64PLUS_3A] Register 1Bh        : 0xFFFF (Interrupt Mask Register)
[C64PLUS_3A] Register 1Ch[00010] : 0x0000 (Spare Control 1)
[C64PLUS_3A] Register 1Ch[00011] : 0x0000 (Clock Alignment Control)
[C64PLUS_3A] Register 1Ch[00100] : 0x000C (Spare Control 2)
[C64PLUS_3A] Register 1Ch[00101] : 0x001F (Spare Control 3)
[C64PLUS_3A] Register 1Ch[01000] : 0x0046 (LED Status)
[C64PLUS_3A] Register 1Ch[01001] : 0x0008 (LED Control)
[C64PLUS_3A] Register 1Ch[01010] : 0x0001 (Auto Power-Down)
[C64PLUS_3A] Register 1Ch[01101] : 0x0010 (LED Selector 1)
[C64PLUS_3A] Register 1Ch[01110] : 0x0063 (LED Selector 2)
[C64PLUS_3A] Register 1Ch[01111] : 0x0000 (LED GPIO Control/Status)
[C64PLUS_3A] Register 1Ch[11000] : 0x0004 (1000BASE-X Auto-Detect SGMII/GBIC)
[C64PLUS_3A] Register 1Ch[11010] : 0x01BF (1000BASE-X Auto-Negotiation Debug)
[C64PLUS_3A] Register 1Ch[11011] : 0x0087 (1000BASE-X Auxiliary Control)
[C64PLUS_3A] Register 1Ch[11100] : 0x021C (1000BASE-X Auxiliary Status)
[C64PLUS_3A] Register 1Ch[11101] : 0x0000 (1000BASE-X Misc Status)
[C64PLUS_3A] Register 1Ch[11110] : 0x0022 (Copper/Fiber Auto-Detect Medium)
[C64PLUS_3A] Register 1Ch[11111] : 0x00EC (Mode Control)
[C64PLUS_3A] ----------------------------------------
[C64PLUS_3A] Network Added: If-1:192.168.1.143
[C64PLUS_3A] Service Status: DHCPC    : Enabled : Running : 017

0 tugrul mutlu over 14 years ago in reply to Alex

Intellectual 365 points

Thank you alex, for your detailed reply..

2 weeks ago i have managed to establish the link and sending packet through ethernet. The problem was about sgmii registers. I think the example configuration is for EVM module and TXN , RXN are connected inversely. When i change it, it establish the link. Again thank you.

Processors

Processors forum

C6474, NDK, Connect Speed