This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

66AK2H14's 1GbE connection to BCM5389

Expert 2985 points
Other Parts Discussed in Thread: 66AK2H14

Hi all,

On my custom board, there are two 66AK2H14s, one BCM5389(Broadcom 8 port 1GbE switch) and one PHY Marvell 88E1111.

The SGMII port 0 of each 66AK2H14 are connected to BCM5389 and the 88E1111's SGMII port is connected to BCM5389 too.

After powering the board on, I find the link between BCM5389 and 88E1111's SGMII port is OK. This link is indicated by one LEDA signal of BCM5389. Low means link is up and If send data through 88E1111 from PC and then this LEDA can blink as the BCM5389's datasheet says. 

So I think the BCM5389 and 88E1111 both work well.

Then I load program into one 66AK2H14. The configuration is as the figure below

And then the following codes can go over.

do
{
	CSL_SGMII_getStatus(macPortNum, &sgmiiStatus);
} while (sgmiiStatus.bIsLinkUp != 1);

/* Wait for SGMII Autonegotiation to complete without error */
do
{
	CSL_SGMII_getStatus(macPortNum, &sgmiiStatus);
	if (sgmiiStatus.bIsAutoNegError != 0)
	    return -1;
} while (sgmiiStatus.bIsAutoNegComplete != 1);

I mean the 66AK2H14 can check the link between itself and BCM5389 is up.

But this link's indication signal LEDA of BCM5389 remains high! I mean the BCM5389 thinks the link between itself and 66AK2H14 is not up.

I think BCM5389 works well.

So is there something wrong in 66AK2H14? But It had checked the SGMII status and found the link is up and the Auto Negotiation is completed.

So why? Can anyone help me?

Regards,

Feng

  • Hi Feng,

    We will check and get back to you as soon as possible. Thank you for your patience.
  • Dear Feng,
    Can you please read the LINK & MR_ADV_ABILITY registers and make sure that LINK is up properly.
    Try to run the NIMU example on your custom board.
    C:\ti\pdk_k2hk_4_0_1\packages\MyExampleProjects\NIMU_emacExample_EVMK2HC66BiosExampleProject

    K2H -> BCM5389 -> 88E1111 -> PC
  • Feng,

    SGMII auto-negotiation requires that one end be the slave (normally the MAC) and one end be the master (normally the PHY).  How is the BCM5389 configured?  The BCM5389 ports are probably MAC ports.  The default configuration for the SGMII ports on the K2H are also as MAC ports.  Both ends cannot be configured as slaves.

    Tom

  • Hi Titusrathinaraj ,

    Yes! I am just debugging this connection K2H -> BCM5389 -> 88E1111 -> PC!

    The MR_ADV_ABILITY reg is read as the figure below

    I set the 66AK2H14's MR_ADV_ABILITY 0x9801 but the MR_LP_ADV_ABILITY(BCM5389's) is 0x4001.

    So is there something wrong?

     And I had run the project C:\ti\pdk_k2hk_4_0_1\packages\MyExampleProjects\NIMU_emacExample_EVMK2HC66BiosExampleProject.

    But when the program run the function NC_NetStart and then the core is stuck as the following picture shows

    So any tips?

    Regards,
    Feng

  • Hi Tom,

    I just want to implement this connection: K2H -> BCM5389 -> 88E1111 -> PC.

    And the BCM5389 is configured as the schematics show BCM5389.pdf

    The link between BCM5389 and 88E1111 is ok.

    BCM5389 works as MAC port.

    So if I want to setup the link between BCM5389 and 66AK2H14, how should I do?

    Regards,

    Feng

  • Feng,

    If the BCM switch port is configured as a MAC (SGMII Slave), then the K2H port must be configured as an SGMII Master.  This is not the standard configuration.  You will need to make modifications to the code to support this.  Please refer to Section 2.4.3 of the Gigabit Ethernet (GbE) Switch Subsystem User Guide (SPRUGV9) on the product page for 66AK2H14 for explanation.

    Tom

  • Hi Tom,

    I had configured the K2H's SGMII port0 as SGMII in Master mode with both auto-negotiation and forced-link as the following pics show.

    And the strange thing is that K2H had polled the SGMII_STATUS as link up, however the BCM5389's link up indication signal LEDA remains high to indicate no link up.

    So why?

    Regards,

    Feng

  • Hi,

    I had checked the MAC Status Register, it shows as below

    Its value is 0x80000000. According to 

    The MAC is in IDLE state. Is it normal??

    My codes are

    Int32 Init_SGMII (UInt32 macPortNum)
    {
    	UInt32					sgmiiCfg_Value 	= 0x9801;
    	UInt32				   *sgmiiCfg_Reg	= (UInt32 *)0x02090118;
        CSL_SGMII_ADVABILITY    sgmiiCfg;
        CSL_SGMII_STATUS        sgmiiStatus;
    
    	/* Reset the port before configuring it */
    	CSL_SGMII_doSoftReset (macPortNum);
    	while (CSL_SGMII_getSoftResetStatus (macPortNum) != 0);
    
    	/* Hold the port in soft reset and set up
    	* the SGMII control register:
    	*      (1) Enable Master Mode (default)
    	*      (2) Enable Auto-negotiation
    	*/
    	CSL_SGMII_startRxTxSoftReset (macPortNum);
    	//CSL_SGMII_disableMasterMode (macPortNum);
    	CSL_SGMII_enableMasterMode (macPortNum);
    
    	/* Setup the Advertised Ability register for this port:
    	*      (1) Enable Full duplex mode
    	*      (2) Enable Auto Negotiation
    	*/
    //	sgmiiCfg.linkSpeed      =   CSL_SGMII_1000_MBPS;
    //	sgmiiCfg.duplexMode     =   CSL_SGMII_FULL_DUPLEX;
    //	//sgmiiCfg.bLinkUp		= 	1;
    //	CSL_SGMII_setAdvAbility (macPortNum, &sgmiiCfg);
    
    	memcpy((void *)sgmiiCfg_Reg, &sgmiiCfg_Value, 4);
    
    	CSL_SGMII_disableAutoNegotiation (macPortNum);
    	//CSL_SGMII_enableAutoNegotiation (macPortNum);
    	CSL_SGMII_endRxTxSoftReset (macPortNum);
    
    	/* Wait for SGMII Link */
    	if (!cpswSimTest)
    	{
    		do
    		{
    			CSL_SGMII_getStatus(macPortNum, &sgmiiStatus);
    		} while (sgmiiStatus.bIsLinkUp != 1);
    
    		/* Wait for SGMII Autonegotiation to complete without error */
    		do
    		{
    			CSL_SGMII_getStatus(macPortNum, &sgmiiStatus);
    			if (sgmiiStatus.bIsAutoNegError != 0)
    				return -1;
    		} while (sgmiiStatus.bIsAutoNegComplete != 1);
    
    		/*
    		 * May need to wait some more time for the external PHY to be ready to transmit packets reliabily.
    		 * It is possible to access the PHY status register through the MDIO interface to check when
    		 * the PHY is ready.
    		 * To avoid platform-dependent code, we just introduce about 2ms wait here
    		 */
    		if((cpswLpbkMode == CPSW_LOOPBACK_EXTERNAL) || (cpswLpbkMode == CPSW_LOOPBACK_NONE))
    			CycleDelay(2000000);
    	}
    
        /* All done with configuration. Return Now. */
        return 0;
    }

    The macportnum is 0.

    Regards,

    Feng

  • Feng,

    If BCM swicth port is MAC, then K2H14 SGMII has to be the master. You can use either master with AN or master with forced link as you tried earlier.

    In your latest code you used memcpy((void *)sgmiiCfg_Reg, &sgmiiCfg_Value, 4); Can you provide the register dump from 0x2090110 to 0x2090120 in both cases, I want to check the CONTROL, STATUS, MR_ADV_ABILITY, and LP_MR_ADV_ABILITY.

    Regarding MACSTATUS 0x2090908, it is 0x80000018 in our EVM where SGMII is the slave. So bit 31 is 1 looks OK.

    What is the SW baseline/release you used? Did you use the same baseline code on any other setups (like TI K2H EVM) to prove the EMAC is working?

    Regards, Eric
  • Hi lding,

    Sorry for the late response!

    SGMII as master(0x9801 to MR_ADV_ABILITY) with AN:

    The registers you mention above are showed as below

    SGMII as master(0x9801 to MR_ADV_ABILITY) without AN and not to poll the auto-negotiation completion:

    My software/hardware situation is: CCSv6.1.3.00034, processor_sdk_rtos_k2hk_3_00_00_04, 66AK2H14's SGMII port0 connection to BCM5389's SGMII port0.

    Unfortunately, I have no K2H EVM and another verified board using 66AK2H14.

    And my whole project is here. SOC1_K2_EMAC.zip

    And more hardware details: I used PASSCLK(100MHz) as the PASS PLL input, not the SYSCLK and not the 122.88MHz as the EVMK2H does.

    So please help me.

    Regards,

    Feng

  • Hi lding,

    I had some new findings.

    BCM5389's serial port has two mode: SGMII mode and SerDes mode.

    And it is SGMII mode as default. Should I change the mode to SerDes mode?? What's the difference?

    Regards,

    Feng

  • Feng,

    SGMII mode and SERDES mode both use the same baud rate and symbols.  As stated in your post, the negotiation is different.  The SERDES mode is for fixed data rate (1000Mbps) whereas the SGMII can negotiate lower data rates with a PHY.  The 66AK2H14 can also be programmed to forced mode to support the 1000BASE-X for fiber interfaces.  However, for your application, we recommend staying in SGMII mode and debugging the software.

    Tom

  • Hi Tom,

    Great! Actually we had a project using C6670 to connect one BCM5389 also with SGMII mode and nothing wrong happened.

    But now we use this 66AK2H14 and the BCM5389 part is the same as what we did before. And then problems come.

    Now I am trying to read the BCM5389' register to find something more. If I have sone valuable findings, I will give you feedback.

    Regards,
    Feng
  • Feng,

    I looked at the SGMII register dump, they looked OK and showed link is up. I need to check the SERDES level registers: can you dump 0x232bfe0/4/8/c and 0x232bff4 to me? And you are using 4 lanes of SGMII?

    Our K2H EVM use:

    • 122.88 for SYSCLK
    • 122.88 for PA
    • 156.25 for SGMII SERDES

    What is yours:

    • ? SYSCLK
    • 100.00 for PA
    • ? SGMII SERDES

    In your code Init_SGMII_SERDES(), it implies that CSL_SERDES_REF_CLOCK_156p25M, need your confirmation.

    Regards, Eric 

  • Hi lding,

    On my custom board,

    100MHz SYSCLK

    100MHz PASS CLK

    156.25MHz SRIO/SGMII SerDes

    And one 66AK2H14's SGMII port0 is connected to BCM5389 and the other three SGMII port is connected to nothing. I just used one lane, the port0. And the other three lanes are set as loopback mode.

    And the registers you mentioned are showed as follow

    I had checked the datasheet. Why can't I find the offset of these registers, both in 66AK2H14 data manual, GBE data manaul and Keystone II's SerDes data manual? In Keystone I, there registers are in Device State Control Registers in C6670's data manual.

    Keysotne II's SerDes looks different from Keystone I's very much.

    Regards,

    Feng

  • Hi lding,

    Additional information...

    If I set the serdes_lane_enable_params.num_lanes = 1, I will get this

    If I set serdes_lane_enable_params.num_lanes more than 1 and don't set the lane 2\3\4 as loopback_disable mode, the program will crash after executing this function CSL_SerdesLaneEnable_Lane_Init_RX.

    Previously, I thought the lane represented each one port of SGMII in 66AK2H14. 4 lanes equals to 4 SGMII port. But it looks like SerDes PHY-A and PHY-A only has 4 lane or 2 lane as the pic shows. I am confused about this.

    Regards,

    Feng

  • Feng,

    SGMII is 4-lane PHY-A, this is explained in Serdes UG. The base of SGMII is 0x232a000, explained in K2H14 datasheet, memory map. The offset 0x1fe0/4/8/c (LANExCTL_STS) and 0x1ff4 (PLL_CTRL).... please check Serdes UG.

    It is right that there are four lanes, each representing a SGMII port. As you just used the first lane, you need to use:

    serdes_lane_enable_params.num_lanes = 4;
    serdes_lane_enable_params.lane_mask = 0x1;

    Will this setting make the program crash?

    Regards, Eric
  • Hi lding,

    I mean if my settings are below

    serdes_lane_enable_params.num_lanes = 4;
    
    serdes_lane_enable_params.loopback_mode[0~3] = CSL_SERDES_LOOPBACK_DISABLED;
    
    serdes_lane_enable_params.lane_mask = (1 << serdes_lane_enable_params.num_lanes) - 1;

    Then program crashed.

    Now my codes are below:

    Int32 Init_SGMII_SERDES(Void)
    {
        if(!cpswSimTest)
        {
            CSL_SERDES_RESULT   csl_retval;
            CSL_SERDES_LANE_ENABLE_STATUS lane_retval = CSL_SERDES_LANE_ENABLE_NO_ERR;
            CSL_SERDES_LANE_ENABLE_PARAMS_T serdes_lane_enable_params;
            uint32_t i;
    
            memset(&serdes_lane_enable_params, 0, sizeof(serdes_lane_enable_params));
    
            serdes_lane_enable_params.base_addr = CSL_NETCP_SERDES_CFG_REGS;
            serdes_lane_enable_params.ref_clock = CSL_SERDES_REF_CLOCK_156p25M;
            serdes_lane_enable_params.linkrate = CSL_SERDES_LINK_RATE_1p25G;
            serdes_lane_enable_params.num_lanes = 4;
            serdes_lane_enable_params.phy_type = SERDES_SGMII;
            for(i=0; i< serdes_lane_enable_params.num_lanes; i++)
            {
                serdes_lane_enable_params.lane_ctrl_rate[i] = CSL_SERDES_LANE_QUARTER_RATE;
                //if(i == 0)
                	serdes_lane_enable_params.loopback_mode[i] = CSL_SERDES_LOOPBACK_DISABLED;
                //else
                	//serdes_lane_enable_params.loopback_mode[i] = CSL_SERDES_LOOPBACK_ENABLED;
    
                /* When RX auto adaptation is on, these are the starting values used for att, boost adaptation */
                serdes_lane_enable_params.rx_coeff.att_start[i] = 7;
                serdes_lane_enable_params.rx_coeff.boost_start[i] = 5;
    
                /* For higher speeds PHY-A, force attenuation and boost values  */
                serdes_lane_enable_params.rx_coeff.force_att_val[i] = 1;
                serdes_lane_enable_params.rx_coeff.force_boost_val[i] = 1;
    
                /* CM, C1, C2 are obtained through Serdes Diagnostic BER test */
                serdes_lane_enable_params.tx_coeff.cm_coeff[i] = 0;
                serdes_lane_enable_params.tx_coeff.c1_coeff[i] = 0;
                serdes_lane_enable_params.tx_coeff.c2_coeff[i] = 0;
                serdes_lane_enable_params.tx_coeff.tx_att[i] = 12;
                serdes_lane_enable_params.tx_coeff.tx_vreg[i] = 4;
            }
    
            /* if the system has 4 lanes, lane mask = 0xF. If the system has 2 lanes, lane mask = 0x3 etc */
            //serdes_lane_enable_params.lane_mask = (1 << serdes_lane_enable_params.num_lanes) - 1;
            serdes_lane_enable_params.lane_mask = 0x01;
        	serdes_lane_enable_params.operating_mode = CSL_SERDES_FUNCTIONAL_MODE;
            /* Att and Boost values are obtained through Serdes Diagnostic PRBS calibration test */
            /* For higher speeds PHY-A, force attenuation and boost values  */
            serdes_lane_enable_params.forceattboost = CSL_SERDES_FORCE_ATT_BOOST_DISABLED;
    
            /* CMU, COMLANE, and Lane Setup  */
            csl_retval = CSL_EthernetSerdesInit(serdes_lane_enable_params.base_addr,
                                                serdes_lane_enable_params.ref_clock,
                                                serdes_lane_enable_params.linkrate);
    
            if (csl_retval != 0)
            {
                System_printf ("Invalid Serdes Init Params\n");
            }
    
            /* Common Init Mode */
            /* Iteration Mode needs to be set to Common Init Mode first */
            serdes_lane_enable_params.iteration_mode = CSL_SERDES_LANE_ENABLE_COMMON_INIT;
            lane_retval = CSL_SerdesLaneEnable(&serdes_lane_enable_params);
            if (lane_retval != 0)
            {
                System_printf ("Invalid Serdes Common Init\n");
                exit(0);
            }
            System_printf("SGMII Serdes Common Init Complete\n");
    
            /* Lane Init Mode */
            /* Once CSL_SerdesLaneEnable is called with iteration_mode = CSL_SERDES_LANE_ENABLE_COMMON_INIT, the lanes needs to be enabled by setting
            iteration_mode =  CSL_SERDES_LANE_ENABLE_LANE_INIT */
            serdes_lane_enable_params.iteration_mode = CSL_SERDES_LANE_ENABLE_LANE_INIT;
            lane_retval = CSL_SerdesLaneEnable(&serdes_lane_enable_params);
            if (lane_retval != 0)
            {
                System_printf ("Invalid Serdes Lane Enable Init\n");
                exit(0);
            }
            System_printf("SGMII Serdes Lanes Init Complete\n");
    
        }
    
        /* SGMII SERDES Configuration complete. Return. */
        return 0;
    }
    Int32 Init_SGMII (UInt32 macPortNum)
    {
    	UInt32					sgmiiCfg_Value 	= 0x9801;
    	UInt32				   *sgmiiCfg_Reg	= (UInt32 *)0x02090118;
        CSL_SGMII_ADVABILITY    sgmiiCfg;
        CSL_SGMII_STATUS        sgmiiStatus;
    
    	/* Reset the port before configuring it */
    	CSL_SGMII_doSoftReset (macPortNum);
    	while (CSL_SGMII_getSoftResetStatus (macPortNum) != 0);
    
    	/* Hold the port in soft reset and set up
    	* the SGMII control register:
    	*      (1) Enable Master Mode (default)
    	*      (2) Enable Auto-negotiation
    	*/
    	CSL_SGMII_startRxTxSoftReset (macPortNum);
    	//CSL_SGMII_disableMasterMode (macPortNum);
    	CSL_SGMII_enableMasterMode (macPortNum);
    
    	/* Setup the Advertised Ability register for this port:
    	*      (1) Enable Full duplex mode
    	*      (2) Enable Auto Negotiation
    	*/
    //	sgmiiCfg.linkSpeed      =   CSL_SGMII_1000_MBPS;
    //	sgmiiCfg.duplexMode     =   CSL_SGMII_FULL_DUPLEX;
    //	//sgmiiCfg.bLinkUp		= 	1;
    //	CSL_SGMII_setAdvAbility (macPortNum, &sgmiiCfg);
    
    	memcpy((void *)sgmiiCfg_Reg, &sgmiiCfg_Value, 4);
    
    	//CSL_SGMII_disableAutoNegotiation (macPortNum);
    	CSL_SGMII_enableAutoNegotiation (macPortNum);
    	CSL_SGMII_endRxTxSoftReset (macPortNum);
    
    	/* Wait for SGMII Link */
    	if (!cpswSimTest)
    	{
    		do
    		{
    			CSL_SGMII_getStatus(macPortNum, &sgmiiStatus);
    		} while (sgmiiStatus.bIsLinkUp != 1);
    
    		/* Wait for SGMII Autonegotiation to complete without error */
    		do
    		{
    			CSL_SGMII_getStatus(macPortNum, &sgmiiStatus);
    			if (sgmiiStatus.bIsAutoNegError != 0)
    				return -1;
    		} while (sgmiiStatus.bIsAutoNegComplete != 1);
    
    		/*
    		 * May need to wait some more time for the external PHY to be ready to transmit packets reliabily.
    		 * It is possible to access the PHY status register through the MDIO interface to check when
    		 * the PHY is ready.
    		 * To avoid platform-dependent code, we just introduce about 2ms wait here
    		 */
    		if((cpswLpbkMode == CPSW_LOOPBACK_EXTERNAL) || (cpswLpbkMode == CPSW_LOOPBACK_NONE))
    			CycleDelay(2000000);
    	}
    
        /* All done with configuration. Return Now. */
        return 0;
    }

    Then I got the following results.

    The 66Ak2H14 had checked the link up but the BCM5389 had not..

    So are there something I need to debug further on 66Ak2H14's side? Or maybe I must check the BCM5389? BCM5389's port7 is connected to PHY 88E1111 and works well.

    Could you use my project here to run on EVMK2H?1651.SOC1_K2_EMAC.zip

    Regards,

    Feng

  • I ran the NIMU example on our K2H EVM, which has the PHY as the master and MAC is the slave. I have the following:

    serdes_lane_enable_params.num_lanes = 2; /* this was originally set to 4 but caused application to not come up */

    serdes_lane_enable_params.loopback_mode[i] = CSL_SERDES_LOOPBACK_DISABLED; for both lanes 0 and 1

    serdes_lane_enable_params.lane_mask = 0x3;

    It worked for me to communicate with outside world.

    Your HW is different from ours, and I can't directly run your code.

    Regards, Eric

  • Hi lding,

    I had load the program NIMU_emacExample_EVMK2KC66BiosExampleProject.

    But I cann't find the codes to configure the SerDes. This NIMU example isn't same as the project helloWorld_evmc6670 which has function platform_init to initialize the SerDes.

    How  NIMU_emacExample_EVMK2KC66BiosExampleProject does?

  • The NIMU example uses helloWorld.cfg, where it contains:
    var socType = "k2h";
    var Nimu = xdc.loadPackage('ti.transport.ndk.nimu');
    Nimu.Settings.socType = socType;
    /*

    So, it uses the NIMU library, the source code is under ti\pdk_k2hk_4_0_1\packages\ti\transport\ndk\nimu\src. V2 folder is for K2H/K device, there is a function called CSL_SgmiiDefSerdesSetup(). This is what you are looking for.

    Regards, Eric
  • Hi lding,

    By the way, I haven't configured the DDR3, I mean the DDR3A/B don't work well. Is this a problem caused what I met?

    Regards,
    Feng
  • Feng,

    I think the problem of SGMII link and DDR3A/B are seperate. If you didn't configure DDR, please don't use DDR in your program, that is, use local L2 or MSMC only.

    Regards, Eric

  • Hi lding,

    My mistake!!
    The 66AK2H14's codes are OK.
    The AUTO_PLL control signal of BCM5389 affects the results!
    Initially, I had set this signal as low to poll PHY then I met the problem.
    Now I set this signal as high no yo poll PHY and then everything goes well!
    Thanks for your consistent help!

    Regards,
    Feng