C6678 SRIO External Loopback

Chris Thunes

I've been trying to get a working external loopback test program for the EVM6678 based on the loopback example included in the mcsdk. I've stripped down the loopback example to just run the blocking test and changed the link speed to 3.125Gbps to match my switch's configuration (CSL_BootCfgSetSRIOSERDESConfigPLL(0x229)).

This works fine as a loopback test, but changing the relevant lines in device_srio_loopback.c to configure the device to run in normal mode hasn't been enough to get the external loopback working. Specifcally I've changed the CSL_BootCfgSetSRIOSERDESTxConfig and CSL_BootCfgSetSRIOSERDESRxConfig calls for non-loopback operation, and changed the existing calls to CSL_SRIO_SetLoopbackMode to calls to CSL_SRIO_SetNormalMode.

The device initializes alright (makes it past the PortOk checks), and the producer thread passes, but the consumer thread hangs waiting for data. My switch is statically configured with the relevant IDs (0xbeef and 0x4560) routed to the port the card is on.

Am I missing anything here, or should this be enough for it to run?

- Chris

over 14 years ago

0 jsobota over 14 years ago

TI__Intellectual 995 points

Chris,

I'm able to run the multicore loopback example under internal and external loopback conditions. The only changes I made to go to external loopback were switching the Tx/RxConfig register values and using NormalMode instead of LoopbackMode in the SrioDevice_init function. I did not even have to change the PLL setting, even though it is outdated.

How are you setting up the external SRIO connections? Do you have an AMC breakout board or AMC chassis? I assume you've got some external connection set up if you're PortOk checks are passing.

Justin

0 Chris Thunes over 14 years ago in reply to jsobota

Intellectual 285 points

Thanks for the quick response Justin.

I'm using the card in a Vadatech AMC chassis with a SRIO MCH also from Vadatech. Because of compatibility issues between the EVM and the chassis, I have the card forcefully powered up using an autonomous power sequencing option on the power module. I also have to manually enable the switch ports. The loopback test hangs waiting for the port to be ready until I manually enable the ports on the switch so I believe there is at least some connectivity between the two.

The only other modficition to the SrioDevice_init I made was to disable ports 1-3 (as well as the PortOk checks) since the MCH only has a single port routed to each AMC slot.

Vadatech was able to help me with querying the value of some registers in the switch from their interface. In particular the switch packet counter stays at 0 during the test which suggests to me that the switch isn't receiving any packets from the EVM.

Thanks again,

Chris

0 NARESH PATEL over 14 years ago

Intellectual 325 points

Hi Chris,

I am also using c6678 EVM . I have compiled srio loopback from <PDK_INSTALL_PATH>\ti\drv\exampleProjects\SRIO_LoopbackTestProject . It compiled successfully. But it failed at run time. Whenever i run it ,it found following error,

**************************************************
******* Ethernet Single Core Example Start *******
**************************************************
Error Initializing Free Descriptors, Error: -131
QMSS init failed

How can i resolved this problem ? have you found such type of problem ? Note that I have load gel file and use ccs 4.2.3 for it.

Thanks.

-Naresh

0 jsobota over 14 years ago in reply to NARESH PATEL

TI__Intellectual 995 points

Chris,

Can you attach your edited device_srio_loopback.c? Until then I'm checking whether disabling 3 of the 4 lanes can affect the loopback test.

Naresh,

What version of MCSDK/PDK are you using? CCSv4 is not supported on MCSDK beta2 release. Please try updating to CCSv5, recompiling, and rerunning the example.

Justin

0 Chris Thunes over 14 years ago in reply to jsobota

Intellectual 285 points

Thanks Justin, device_srio_loopback.c attached.

Fullscreen 0250.device_srio_loopback.c.txt Download

/**
 *   @file  device_srio_loopback.c
 *
 *   @brief   
 *      The 6616 SRIO Device specific code. The SRIO driver calls out
 *      this code to initialize the SRIO IP block. The file is provided as 
 *      a sample configuration and should be modified by customers for 
 *      their own platforms and configurations.
 *
 *  \par
 *  NOTE:
 *      (C) Copyright 2009 Texas Instruments, Inc.
 *  \par
 */

/* SRIO Driver Includes. */
#include <ti/drv/srio/srio_types.h>
#include <ti/drv/srio/include/listlib.h>
#include <ti/drv/srio/srio_drv.h>

/* CSL SRIO Functional Layer */
#include <ti/csl/csl_srio.h>
#include <ti/csl/csl_srioAux.h>
#include <ti/csl/csl_srioAuxPhyLayer.h>

/* CSL BootCfg Module */
#include <ti/csl/csl_bootcfg.h>
#include <ti/csl/csl_bootcfgAux.h>

/* CSL Chip Functional Layer */
#include <ti/csl/csl_chip.h>

/* QMSS Include */
#include <ti/drv/qmss/qmss_drv.h>

/**********************************************************************
 ************************* LOCAL Definitions **************************
 **********************************************************************/

/* These are the GARBAGE queues which are used by the TXU to dump the 
 * descriptor if there is an error instead of recycling the descriptor
 * to the free queue. */
#define GARBAGE_LEN_QUEUE		    905
#define GARBAGE_TOUT_QUEUE		    906
#define GARBAGE_RETRY_QUEUE		    907
#define GARBAGE_TRANS_ERR_QUEUE	    908
#define GARBAGE_PROG_QUEUE		    909
#define GARBAGE_SSIZE_QUEUE		    910

/* SRIO Device Information
 * - 16 bit Device Identifier.
 * - 8 bit Device Identifier.
 * - Vendor Identifier. 
 * - Device Revision. */
#define DEVICE_VENDOR_ID            0x30
#define DEVICE_REVISION             0x0

/* SRIO Assembly Information
 * - Assembly Identifier
 * - Assembly Vendor Identifier. 
 * - Assembly Device Revision. 
 * - Assembly Extension Features */
#define DEVICE_ASSEMBLY_ID          0x0
#define DEVICE_ASSEMBLY_VENDOR_ID   0x30
#define DEVICE_ASSEMBLY_REVISION    0x0
#define DEVICE_ASSEMBLY_INFO        0x0100

/**********************************************************************
 ************************* Extern Definitions *************************
 **********************************************************************/

extern const uint32_t DEVICE_ID1_16BIT;
extern const uint32_t DEVICE_ID1_8BIT;
extern const uint32_t DEVICE_ID2_16BIT;
extern const uint32_t DEVICE_ID2_8BIT;
extern const uint32_t DEVICE_ID3_16BIT;
extern const uint32_t DEVICE_ID3_8BIT_ID;
extern const uint32_t DEVICE_ID4_16BIT;
extern const uint32_t DEVICE_ID4_8BIT_ID;

#define PORT_COUNT 1

/**********************************************************************
 *********************** DEVICE SRIO FUNCTIONS ***********************
 **********************************************************************/

/** @addtogroup SRIO_DEVICE_API
 @{ */

/**
 *  @b Description
 *  @n  
 *      The function provides the initialization sequence for the SRIO IP
 *      block. This can be modified by customers for their application and
 *      configuration.
 *
 *  @retval
 *      Success     -   0
 *  @retval
 *      Error       -   <0
 */
int32_t SrioDevice_init (void)
{
    CSL_SrioHandle      hSrio;
	int32_t             i;
    SRIO_PE_FEATURES    peFeatures;
    SRIO_OP_CAR         opCar;
    Qmss_QueueHnd       queueHnd;
    uint8_t             isAllocated;
    uint32_t            gargbageQueue[] = { GARBAGE_LEN_QUEUE,  GARBAGE_TOUT_QUEUE,
                                            GARBAGE_RETRY_QUEUE,GARBAGE_TRANS_ERR_QUEUE,
                                            GARBAGE_PROG_QUEUE, GARBAGE_SSIZE_QUEUE };

    /* Get the CSL SRIO Handle. */
    hSrio = CSL_SRIO_Open (0);
    if (hSrio == NULL)
        return -1;
 
    /* Disable the SRIO Global block */
   	CSL_SRIO_GlobalDisable (hSrio);
   	
   	/* Disable each of the individual SRIO blocks. */
   	for(i = 0; i <= 9; i++)
   		CSL_SRIO_DisableBlock(hSrio, i);

    /* Set boot complete to be 0; we are not done with the initialization. */	
	CSL_SRIO_SetBootComplete(hSrio, 0);

    /* Now enable the SRIO block and all the individual blocks also. */
    CSL_SRIO_GlobalEnable (hSrio);
    for(i = 0; i <= 9; i++)
        CSL_SRIO_EnableBlock(hSrio,i);

    /* Configure SRIO ports to operate in loopback mode. */
    for(i = 0; i < PORT_COUNT; i++) {
    	CSL_SRIO_SetNormalMode(hSrio, i);
    }

	/* Enable Automatic Priority Promotion of response packets. */
	CSL_SRIO_EnableAutomaticPriorityPromotion(hSrio);

	/* Set the SRIO Prescalar select to operate in the range of 44.7 to 89.5 */
	CSL_SRIO_SetPrescalarSelect (hSrio, 0);

    /* Unlock the Boot Configuration Kicker */
    CSL_BootCfgUnlockKicker ();

    /* Assuming the link rate is 3.125; program the PLL accordingly. */
    CSL_BootCfgSetSRIOSERDESConfigPLL (0x229);

    for(i = 0; i < PORT_COUNT; i++) {
        /* Configure the SRIO SERDES Receive Configuration. */
    	CSL_BootCfgSetSRIOSERDESRxConfig (i, 0x000440495);

    	/* Configure the SRIO SERDES Transmit Configuration. */
    	CSL_BootCfgSetSRIOSERDESTxConfig (i, 0x00180795);
    }

#ifndef SIMULATOR_SUPPORT
    /* Loop around till the SERDES PLL is not locked. */
    while (1)
    {
        uint32_t    status;

        /* Get the SRIO SERDES Status */
        CSL_BootCfgGetSRIOSERDESStatus(&status);
        if (status & 0x1)
            break;
    }
#endif

    /* Clear the LSU pending interrupts. */
    CSL_SRIO_ClearLSUPendingInterrupt (hSrio, 0xFFFFFFFF, 0xFFFFFFFF);

    /* Set the Device Information */
    CSL_SRIO_SetDeviceInfo (hSrio, DEVICE_ID1_16BIT, DEVICE_VENDOR_ID, DEVICE_REVISION);

    /* Set the Assembly Information */
    CSL_SRIO_SetAssemblyInfo(hSrio, DEVICE_ASSEMBLY_ID, DEVICE_ASSEMBLY_VENDOR_ID, 
                             DEVICE_ASSEMBLY_REVISION, DEVICE_ASSEMBLY_INFO);

    /* TODO: Configure the processing element features
     *  The SRIO RL file is missing the Re-transmit Suppression Support (Bit6) field definition */
    peFeatures.isBridge                          = 0;
    peFeatures.isEndpoint                        = 0;
    peFeatures.isProcessor                       = 1;
    peFeatures.isSwitch                          = 0;
    peFeatures.isMultiport                       = 0;
    peFeatures.isFlowArbiterationSupported       = 0;
    peFeatures.isMulticastSupported              = 0;
    peFeatures.isExtendedRouteConfigSupported    = 0;
    peFeatures.isStandardRouteConfigSupported    = 1;
    peFeatures.isFlowControlSupported            = 1;
    peFeatures.isCRFSupported                    = 0;
    peFeatures.isCTLSSupported                   = 1;
    peFeatures.isExtendedFeaturePtrValid         = 1;
    peFeatures.numAddressBitSupported            = 1;
    CSL_SRIO_SetProcessingElementFeatures (hSrio, &peFeatures);

    /* Configure the source operation CAR */
    memset ((void *) &opCar, 0, sizeof (opCar));
    opCar.portWriteOperationSupport = 1;
    opCar.atomicClearSupport        = 1;
    opCar.atomicSetSupport          = 1;
    opCar.atomicDecSupport          = 1;
    opCar.atomicIncSupport          = 1;
    opCar.atomicTestSwapSupport     = 1;
    opCar.doorbellSupport           = 1;
    opCar.dataMessageSupport        = 1;
    opCar.writeResponseSupport      = 1;
    opCar.streamWriteSupport        = 1;
    opCar.writeSupport              = 1;
    opCar.readSupport               = 1;
    opCar.dataStreamingSupport      = 1;
    CSL_SRIO_SetSourceOperationCAR (hSrio, &opCar);

    /* Configure the destination operation CAR */
	memset ((void *) &opCar, 0, sizeof (opCar));
    opCar.portWriteOperationSupport  = 1;
    opCar.doorbellSupport            = 1;
    opCar.dataMessageSupport         = 1;
    opCar.writeResponseSupport       = 1;
    opCar.streamWriteSupport         = 1;
    opCar.writeSupport               = 1;
    opCar.readSupport                = 1;
    CSL_SRIO_SetDestOperationCAR (hSrio, &opCar);

    /* Set the 16 bit and 8 bit identifier for the SRIO Device. */
    CSL_SRIO_SetDeviceIDCSR (hSrio, DEVICE_ID1_8BIT, DEVICE_ID1_16BIT);

    /* Enable TLM Base Routing Information for Maintainance Requests & ensure that
     * the BRR's can be used by all the ports. */
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 0, 1, 1, 1, 0);
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 0, 2, 1, 1, 0);
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 0, 3, 1, 1, 0);
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 1, 0, 1, 1, 0);

    /* Configure the Base Routing Register to ensure that all packets matching the 
     * Device Identifier & the Secondary Device Id are admitted. */
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 0, 1, DEVICE_ID2_16BIT, 0xFFFF);
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 0, 2, DEVICE_ID3_16BIT, 0xFFFF);
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 0, 3, DEVICE_ID4_16BIT, 0xFFFF);
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 1, 0, DEVICE_ID2_8BIT,  0xFF);

    /* We need to open the Garbage collection queues in the QMSS. This is done to ensure that 
     * these queues are not opened by another system entity. */
    for (i = 0; i < 6; i++)
    {
        /* Open the Garabage queues */
        queueHnd = Qmss_queueOpen (Qmss_QueueType_GENERAL_PURPOSE_QUEUE, gargbageQueue[i], &isAllocated);
        if (queueHnd < 0)
            return -1;

        /* Make sure the queue has not been opened already; we dont the queues to be shared by some other
         * entity in the system. */
        if (isAllocated > 1)
            return -1;
    }

    /* Set the Transmit Garbage Collection Information. */
    CSL_SRIO_SetTxGarbageCollectionInfo (hSrio, GARBAGE_LEN_QUEUE, GARBAGE_TOUT_QUEUE, 
                                         GARBAGE_RETRY_QUEUE, GARBAGE_TRANS_ERR_QUEUE, 
                                         GARBAGE_PROG_QUEUE, GARBAGE_SSIZE_QUEUE);

    /* Set the Host Device Identifier. */
    CSL_SRIO_SetHostDeviceID (hSrio, DEVICE_ID1_16BIT);

    /* Configure the component tag CSR */
    CSL_SRIO_SetCompTagCSR (hSrio, 0x00000000);

    /* Configure the PLM for all the ports. */
	for (i = 0; i < PORT_COUNT; i++)
	{
	    /* Set the PLM Port Silence Timer. */	
        CSL_SRIO_SetPLMPortSilenceTimer (hSrio, i, 0x2);

        /* TODO: We need to ensure that the Port 0 is configured to support both
         * the 2x and 4x modes. The Port Width field is read only. So here we simply
         * ensure that the Input and Output ports are enabled. */
        CSL_SRIO_EnableInputPort (hSrio, i);
        CSL_SRIO_EnableOutputPort (hSrio, i);

        /* Set the PLM Port Discovery Timer. */
        CSL_SRIO_SetPLMPortDiscoveryTimer (hSrio, i, 0x2);

        /* Reset the Port Write Reception capture. */
        CSL_SRIO_SetPortWriteReceptionCapture(hSrio, i, 0x0);
    }

    /* Set the Port link timeout CSR */
    CSL_SRIO_SetPortLinkTimeoutCSR (hSrio, 0x000FFF);

    /* Set the Port General CSR: Only executing as Master Enable */
    CSL_SRIO_SetPortGeneralCSR (hSrio, 0, 1, 0);

    /* Clear the sticky register bits. */
    CSL_SRIO_SetLLMResetControl (hSrio, 1);

    /* Set the device id to be 0 for the Maintenance Port-Write operation 
     * to report errors to a system host. */
    CSL_SRIO_SetPortWriteDeviceId (hSrio, 0x0, 0x0, 0x0);

    /* Set the Data Streaming MTU */
    CSL_SRIO_SetDataStreamingMTU (hSrio, 64);

    /* Configure the path mode for the ports. */
    for(i = 0; i < PORT_COUNT; i++)
        CSL_SRIO_SetPLMPortPathControlMode (hSrio, i, 0);

    /* Set the LLM Port IP Prescalar. */
    CSL_SRIO_SetLLMPortIPPrescalar (hSrio, 0x21);

    /* Enable the peripheral. */
    CSL_SRIO_EnablePeripheral(hSrio);

    /* Configuration has been completed. */
    CSL_SRIO_SetBootComplete(hSrio, 1);

#ifndef SIMULATOR_SUPPORT
    /* This code checks if the ports are operational or not. The functionality is not supported 
     * on the simulator. */    
	for(i = 0; i < PORT_COUNT; i++)
        while (CSL_SRIO_IsPortOk (hSrio, i) != TRUE);
#endif

    /* Set all the queues 0 to operate at the same priority level and to send packets onto Port 0 */
    for (i =0 ; i < 16; i++)
        CSL_SRIO_SetTxQueueSchedInfo(hSrio, i, 0, 0);

    /* Set the Doorbell route to determine which routing table is to be used 
     * This configuration implies that the Interrupt Routing Table is configured as 
     * follows:-
     *  Interrupt Destination 0 - INTDST 16 
     *  Interrupt Destination 1 - INTDST 17 
     *  Interrupt Destination 2 - INTDST 18
     *  Interrupt Destination 3 - INTDST 19 
     */
    CSL_SRIO_SetDoorbellRoute(hSrio, 0);

    /* Route the Doorbell interrupts. 
     *  Doorbell Register 0 - All 16 Doorbits are routed to Interrupt Destination 0. 
     *  Doorbell Register 1 - All 16 Doorbits are routed to Interrupt Destination 1. 
     *  Doorbell Register 2 - All 16 Doorbits are routed to Interrupt Destination 2. 
     *  Doorbell Register 3 - All 16 Doorbits are routed to Interrupt Destination 3. */
    for (i = 0; i < 16; i++)
    {
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 0, i, 0);
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 1, i, 1);
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 2, i, 2);
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 3, i, 3);
    }

    /* Initialization has been completed. */
    return 0;
}

/**
@}
*/

- Chris

0 Chris Thunes over 14 years ago in reply to Chris Thunes

Intellectual 285 points

Hi Justin,

Are there any updates on this issue?

Thanks,

Chris

0 jsobota over 14 years ago in reply to Chris Thunes

TI__Intellectual 995 points

Chris,

Sorry, for the delay, I'm going to run some tests using your device.c. I'll get back to you by the evening.

Justin

0 jsobota over 14 years ago in reply to jsobota

TI__Intellectual 995 points

Chris,

I ran the following tests using the multicore loopback example provided in PDK 6678 1.0.0.9 beta2 :

1. Internal loopback enabled (project built as-is) - SUCCESSFUL on all four cores

On breakout board only looped back on port 0, ports 1-3 were left disconnected

2. Set normalMode, SERDES registers to non-loopback, and configured the PLL to 0x241 for 3.25Gbps - FAILED - no ports synced

3. Only check to see if port 0 is OK during execution of SrioDevice_init - FAILED - no ports synced

4. Used your device_srio as-is - FAILED - no ports synced

On breakout board looped back on all ports, 0 - 3

5. Reran test 2. - SUCCESSFUL on all four cores

6. Reran test 3. - SUCCESSFUL on all four cores

7. Reran test 4. - SUCCESSUL on core 2 only. Core's 0, 1, and 3 are all in the "Waiting for data to arrive" state. printout follows:

Unfortunately, until next week I don't have much time to dig any deeper than this. It seems that despite your changes, some data is still being routed over the other ports. Also, given I'm able to perform a successful transfer using your device_srio.c from core 1 to core 2 I think there's still an issue with your switch.

Next week I can thoroughly review the settings made in device_srio.c and attempt to boil it down to configuring, and routing all traffic over a single port.

Justin

0 Chris Thunes over 14 years ago in reply to jsobota

Intellectual 285 points

Hi Justin,

Thanks for looking into this for me. If you get a chance before next week, could you give the following a shot on your single port loopback test setup? I've done what I can do disable all the other ports. It doesn't work on my setup, but if I have an issue with my switch I thought it may work in yours.

Thanks,

Chris

Fullscreen 3108.device_srio_loopback.c.txt Download

/**
 *   @file  device_srio_loopback.c
 *
 *   @brief   
 *      The 6616 SRIO Device specific code. The SRIO driver calls out
 *      this code to initialize the SRIO IP block. The file is provided as 
 *      a sample configuration and should be modified by customers for 
 *      their own platforms and configurations.
 *
 *  \par
 *  NOTE:
 *      (C) Copyright 2009 Texas Instruments, Inc.
 *  \par
 */

/* SRIO Driver Includes. */
#include <ti/drv/srio/srio_types.h>
#include <ti/drv/srio/include/listlib.h>
#include <ti/drv/srio/srio_drv.h>

/* CSL SRIO Functional Layer */
#include <ti/csl/csl_srio.h>
#include <ti/csl/csl_srioAux.h>
#include <ti/csl/csl_srioAuxPhyLayer.h>

/* CSL BootCfg Module */
#include <ti/csl/csl_bootcfg.h>
#include <ti/csl/csl_bootcfgAux.h>

/* CSL Chip Functional Layer */
#include <ti/csl/csl_chip.h>

/* QMSS Include */
#include <ti/drv/qmss/qmss_drv.h>

/**********************************************************************
 ************************* LOCAL Definitions **************************
 **********************************************************************/

/* These are the GARBAGE queues which are used by the TXU to dump the 
 * descriptor if there is an error instead of recycling the descriptor
 * to the free queue. */
#define GARBAGE_LEN_QUEUE		    905
#define GARBAGE_TOUT_QUEUE		    906
#define GARBAGE_RETRY_QUEUE		    907
#define GARBAGE_TRANS_ERR_QUEUE	    908
#define GARBAGE_PROG_QUEUE		    909
#define GARBAGE_SSIZE_QUEUE		    910

/* SRIO Device Information
 * - 16 bit Device Identifier.
 * - 8 bit Device Identifier.
 * - Vendor Identifier. 
 * - Device Revision. */
#define DEVICE_VENDOR_ID            0x30
#define DEVICE_REVISION             0x0

/* SRIO Assembly Information
 * - Assembly Identifier
 * - Assembly Vendor Identifier. 
 * - Assembly Device Revision. 
 * - Assembly Extension Features */
#define DEVICE_ASSEMBLY_ID          0x0
#define DEVICE_ASSEMBLY_VENDOR_ID   0x30
#define DEVICE_ASSEMBLY_REVISION    0x0
#define DEVICE_ASSEMBLY_INFO        0x0100

/**********************************************************************
 ************************* Extern Definitions *************************
 **********************************************************************/

extern const uint32_t DEVICE_ID1_16BIT;
extern const uint32_t DEVICE_ID1_8BIT;
extern const uint32_t DEVICE_ID2_16BIT;
extern const uint32_t DEVICE_ID2_8BIT;
extern const uint32_t DEVICE_ID3_16BIT;
extern const uint32_t DEVICE_ID3_8BIT_ID;
extern const uint32_t DEVICE_ID4_16BIT;
extern const uint32_t DEVICE_ID4_8BIT_ID;

#define PORT_COUNT 1

/**********************************************************************
 *********************** DEVICE SRIO FUNCTIONS ***********************
 **********************************************************************/

/** @addtogroup SRIO_DEVICE_API
 @{ */

/**
 *  @b Description
 *  @n  
 *      The function provides the initialization sequence for the SRIO IP
 *      block. This can be modified by customers for their application and
 *      configuration.
 *
 *  @retval
 *      Success     -   0
 *  @retval
 *      Error       -   <0
 */
int32_t SrioDevice_init (void)
{
    CSL_SrioHandle      hSrio;
	int32_t             i;
    SRIO_PE_FEATURES    peFeatures;
    SRIO_OP_CAR         opCar;
    Qmss_QueueHnd       queueHnd;
    uint8_t             isAllocated;
    uint32_t            gargbageQueue[] = { GARBAGE_LEN_QUEUE,  GARBAGE_TOUT_QUEUE,
                                            GARBAGE_RETRY_QUEUE,GARBAGE_TRANS_ERR_QUEUE,
                                            GARBAGE_PROG_QUEUE, GARBAGE_SSIZE_QUEUE };

    /* Get the CSL SRIO Handle. */
    hSrio = CSL_SRIO_Open (0);
    if (hSrio == NULL)
        return -1;
 
    /* Disable the SRIO Global block */
   	CSL_SRIO_GlobalDisable (hSrio);
   	
   	/* Disable each of the individual SRIO blocks. */
   	for(i = 0; i <= 9; i++)
   		CSL_SRIO_DisableBlock(hSrio, i);

    /* Set boot complete to be 0; we are not done with the initialization. */	
	CSL_SRIO_SetBootComplete(hSrio, 0);

    /* Now enable the SRIO block and all the individual blocks also. */
    CSL_SRIO_GlobalEnable (hSrio);
    for(i = 0; i <= 5; i++) /* was <= 9 */
        CSL_SRIO_EnableBlock(hSrio,i);

    /* Configure SRIO ports to operate in loopback mode. */
    for(i = 0; i < PORT_COUNT; i++) {
    	CSL_SRIO_SetNormalMode(hSrio, i);
    }

	/* Enable Automatic Priority Promotion of response packets. */
	CSL_SRIO_EnableAutomaticPriorityPromotion(hSrio);

	/* Set the SRIO Prescalar select to operate in the range of 44.7 to 89.5 */
	CSL_SRIO_SetPrescalarSelect (hSrio, 0);

    /* Unlock the Boot Configuration Kicker */
    CSL_BootCfgUnlockKicker ();

    /* Assuming the link rate is 3.125; program the PLL accordingly. */
    CSL_BootCfgSetSRIOSERDESConfigPLL (0x229);

    for(i = 0; i < PORT_COUNT; i++) {
        /* Configure the SRIO SERDES Receive Configuration. */
    	CSL_BootCfgSetSRIOSERDESRxConfig (i, 0x000440495);

    	/* Configure the SRIO SERDES Transmit Configuration. */
    	CSL_BootCfgSetSRIOSERDESTxConfig (i, 0x00180795);
    }

#ifndef SIMULATOR_SUPPORT
    /* Loop around till the SERDES PLL is not locked. */
    while (1)
    {
        uint32_t    status;

        /* Get the SRIO SERDES Status */
        CSL_BootCfgGetSRIOSERDESStatus(&status);
        if (status & 0x1)
            break;
    }
#endif

    /* Clear the LSU pending interrupts. */
    CSL_SRIO_ClearLSUPendingInterrupt (hSrio, 0xFFFFFFFF, 0xFFFFFFFF);

    /* Set the Device Information */
    CSL_SRIO_SetDeviceInfo (hSrio, DEVICE_ID1_16BIT, DEVICE_VENDOR_ID, DEVICE_REVISION);

    /* Set the Assembly Information */
    CSL_SRIO_SetAssemblyInfo(hSrio, DEVICE_ASSEMBLY_ID, DEVICE_ASSEMBLY_VENDOR_ID, 
                             DEVICE_ASSEMBLY_REVISION, DEVICE_ASSEMBLY_INFO);

    /* TODO: Configure the processing element features
     *  The SRIO RL file is missing the Re-transmit Suppression Support (Bit6) field definition */
    peFeatures.isBridge                          = 0;
    peFeatures.isEndpoint                        = 0;
    peFeatures.isProcessor                       = 1;
    peFeatures.isSwitch                          = 0;
    peFeatures.isMultiport                       = 0;
    peFeatures.isFlowArbiterationSupported       = 0;
    peFeatures.isMulticastSupported              = 0;
    peFeatures.isExtendedRouteConfigSupported    = 0;
    peFeatures.isStandardRouteConfigSupported    = 1;
    peFeatures.isFlowControlSupported            = 1;
    peFeatures.isCRFSupported                    = 0;
    peFeatures.isCTLSSupported                   = 1;
    peFeatures.isExtendedFeaturePtrValid         = 1;
    peFeatures.numAddressBitSupported            = 1;
    CSL_SRIO_SetProcessingElementFeatures (hSrio, &peFeatures);

    /* Configure the source operation CAR */
    memset ((void *) &opCar, 0, sizeof (opCar));
    opCar.portWriteOperationSupport = 1;
    opCar.atomicClearSupport        = 1;
    opCar.atomicSetSupport          = 1;
    opCar.atomicDecSupport          = 1;
    opCar.atomicIncSupport          = 1;
    opCar.atomicTestSwapSupport     = 1;
    opCar.doorbellSupport           = 1;
    opCar.dataMessageSupport        = 1;
    opCar.writeResponseSupport      = 1;
    opCar.streamWriteSupport        = 1;
    opCar.writeSupport              = 1;
    opCar.readSupport               = 1;
    opCar.dataStreamingSupport      = 1;
    CSL_SRIO_SetSourceOperationCAR (hSrio, &opCar);

    /* Configure the destination operation CAR */
	memset ((void *) &opCar, 0, sizeof (opCar));
    opCar.portWriteOperationSupport  = 1;
    opCar.doorbellSupport            = 1;
    opCar.dataMessageSupport         = 1;
    opCar.writeResponseSupport       = 1;
    opCar.streamWriteSupport         = 1;
    opCar.writeSupport               = 1;
    opCar.readSupport                = 1;
    CSL_SRIO_SetDestOperationCAR (hSrio, &opCar);

    /* Set the 16 bit and 8 bit identifier for the SRIO Device. */
    CSL_SRIO_SetDeviceIDCSR (hSrio, DEVICE_ID1_8BIT, DEVICE_ID1_16BIT);

    /* Enable TLM Base Routing Information for Maintainance Requests & ensure that
     * the BRR's can be used by all the ports. */
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 0, 1, 1, 1, 0);
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 0, 2, 1, 1, 0);
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 0, 3, 1, 1, 0);
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 1, 0, 1, 1, 0);

    /* Configure the Base Routing Register to ensure that all packets matching the 
     * Device Identifier & the Secondary Device Id are admitted. */
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 0, 1, DEVICE_ID2_16BIT, 0xFFFF);
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 0, 2, DEVICE_ID3_16BIT, 0xFFFF);
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 0, 3, DEVICE_ID4_16BIT, 0xFFFF);
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 1, 0, DEVICE_ID2_8BIT,  0xFF);

    /* We need to open the Garbage collection queues in the QMSS. This is done to ensure that 
     * these queues are not opened by another system entity. */
    for (i = 0; i < 6; i++)
    {
        /* Open the Garabage queues */
        queueHnd = Qmss_queueOpen (Qmss_QueueType_GENERAL_PURPOSE_QUEUE, gargbageQueue[i], &isAllocated);
        if (queueHnd < 0)
            return -1;

        /* Make sure the queue has not been opened already; we dont the queues to be shared by some other
         * entity in the system. */
        if (isAllocated > 1)
            return -1;
    }

    /* Set the Transmit Garbage Collection Information. */
    CSL_SRIO_SetTxGarbageCollectionInfo (hSrio, GARBAGE_LEN_QUEUE, GARBAGE_TOUT_QUEUE, 
                                         GARBAGE_RETRY_QUEUE, GARBAGE_TRANS_ERR_QUEUE, 
                                         GARBAGE_PROG_QUEUE, GARBAGE_SSIZE_QUEUE);

    /* Set the Host Device Identifier. */
    CSL_SRIO_SetHostDeviceID (hSrio, DEVICE_ID1_16BIT);

    /* Configure the component tag CSR */
    CSL_SRIO_SetCompTagCSR (hSrio, 0x00000000);

    /* Configure the PLM for all the ports. */
	for (i = 0; i < PORT_COUNT; i++)
	{
	    /* Set the PLM Port Silence Timer. */	
        CSL_SRIO_SetPLMPortSilenceTimer (hSrio, i, 0x2);

        /* TODO: We need to ensure that the Port 0 is configured to support both
         * the 2x and 4x modes. The Port Width field is read only. So here we simply
         * ensure that the Input and Output ports are enabled. */
        CSL_SRIO_EnableInputPort (hSrio, i);
        CSL_SRIO_EnableOutputPort (hSrio, i);

        /* Set the PLM Port Discovery Timer. */
        CSL_SRIO_SetPLMPortDiscoveryTimer (hSrio, i, 0x2);

        /* Reset the Port Write Reception capture. */
        CSL_SRIO_SetPortWriteReceptionCapture(hSrio, i, 0x0);
    }

	for (i = PORT_COUNT; i < 4; i++) {
		CSL_SRIO_DisableInputPort(hSrio, i);
		CSL_SRIO_DisableOutputPort(hSrio, i);
		CSL_SRIO_DisablePort(hSrio, i);
	}

    /* Set the Port link timeout CSR */
    CSL_SRIO_SetPortLinkTimeoutCSR (hSrio, 0x000FFF);

    /* Set the Port General CSR: Only executing as Master Enable */
    CSL_SRIO_SetPortGeneralCSR (hSrio, 0, 1, 0);

    /* Clear the sticky register bits. */
    CSL_SRIO_SetLLMResetControl (hSrio, 1);

    /* Set the device id to be 0 for the Maintenance Port-Write operation 
     * to report errors to a system host. */
    CSL_SRIO_SetPortWriteDeviceId (hSrio, 0x0, 0x0, 0x0);

    /* Set the Data Streaming MTU */
    CSL_SRIO_SetDataStreamingMTU (hSrio, 64);

    /* Configure the path mode for the ports. */
    for(i = 0; i < PORT_COUNT; i++)
        CSL_SRIO_SetPLMPortPathControlMode (hSrio, i, 0);

    /* Set the LLM Port IP Prescalar. */
    CSL_SRIO_SetLLMPortIPPrescalar (hSrio, 0x21);

    /* Enable the peripheral. */
    CSL_SRIO_EnablePeripheral(hSrio);

    /* Configuration has been completed. */
    CSL_SRIO_SetBootComplete(hSrio, 1);

#ifndef SIMULATOR_SUPPORT
    /* This code checks if the ports are operational or not. The functionality is not supported 
     * on the simulator. */    
	for(i = 0; i < PORT_COUNT; i++)
        while (CSL_SRIO_IsPortOk (hSrio, i) != TRUE);
#endif

    /* Set all the queues 0 to operate at the same priority level and to send packets onto Port 0 */
    for (i =0 ; i < 16; i++)
        CSL_SRIO_SetTxQueueSchedInfo(hSrio, i, 0, 0);

    /* Set the Doorbell route to determine which routing table is to be used 
     * This configuration implies that the Interrupt Routing Table is configured as 
     * follows:-
     *  Interrupt Destination 0 - INTDST 16 
     *  Interrupt Destination 1 - INTDST 17 
     *  Interrupt Destination 2 - INTDST 18
     *  Interrupt Destination 3 - INTDST 19 
     */
    CSL_SRIO_SetDoorbellRoute(hSrio, 0);

    /* Route the Doorbell interrupts. 
     *  Doorbell Register 0 - All 16 Doorbits are routed to Interrupt Destination 0. 
     *  Doorbell Register 1 - All 16 Doorbits are routed to Interrupt Destination 1. 
     *  Doorbell Register 2 - All 16 Doorbits are routed to Interrupt Destination 2. 
     *  Doorbell Register 3 - All 16 Doorbits are routed to Interrupt Destination 3. */
    for (i = 0; i < 16; i++)
    {
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 0, i, 0);
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 1, i, 1);
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 2, i, 2);
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 3, i, 3);
    }

    /* Initialization has been completed. */
    return 0;
}

/**
@}
*/

0 jsobota over 14 years ago in reply to Chris Thunes

TI__Intellectual 995 points

Chris,

I ran your new device_srio.c with all four ports looped back on the breakout board and with only the first port looped back (all others were disconnected completely). Both tests passed for all four cores.

[C66xx_3] Debug(Core 3): Waiting for data to arrive 0x5678
[C66xx_0] ------------------------------------------------------
[C66xx_1] ------------------------------------------------------
[C66xx_2] ------------------------------------------------------
[C66xx_0] Debug(Core 0): Waiting for data to arrive 0xbeef
[C66xx_1] Debug(Core 1): Successfully sent data to ID:0x1234
[C66xx_2] Debug(Core 2): Waiting for data to arrive 0x1234
[C66xx_1] Debug(Core 1): Waiting for data to arrive 0x4560
[C66xx_2] Debug(Core 2): Successfully received 104 bytes
[C66xx_2] Debug(Core 2): Successfully sent data to ID:0x5678
[C66xx_2] Debug(Core 2): Multicore Test Passed
[C66xx_2] Debug(Core 2): Allocation Counter : 16
[C66xx_2] Debug(Core 2): Free Counter       : 7
[C66xx_2] Debug(Core 2): Multicore Example successful.
[C66xx_3] Debug(Core 3): Successfully received 104 bytes
[C66xx_3] Debug(Core 3): Successfully sent data to ID:0xbeef
[C66xx_3] Debug(Core 3): Multicore Test Passed
[C66xx_3] Debug(Core 3): Allocation Counter : 16
[C66xx_3] Debug(Core 3): Free Counter       : 7
[C66xx_3] Debug(Core 3): Multicore Example successful.
[C66xx_0] Debug(Core 0): Successfully received 104 bytes
[C66xx_0] Debug(Core 0): Successfully sent data to ID:0x4560
[C66xx_0] Debug(Core 0): Multicore Test Passed
[C66xx_0] Debug(Core 0): Allocation Counter : 16
[C66xx_0] Debug(Core 0): Free Counter       : 7
[C66xx_0] Debug(Core 0): Multicore Example successful.
[C66xx_1] Debug(Core 1): Successfully received 104 bytes
[C66xx_1] Debug(Core 1): Multicore Test Passed
[C66xx_1] Debug(Core 1): Allocation Counter : 16
[C66xx_1] Debug(Core 1): Free Counter       : 7
[C66xx_1] Debug(Core 1): Multicore Example successful.

It looks like your switch is the problem.

Justin

0 Chris Thunes over 14 years ago in reply to jsobota

Intellectual 285 points

Thanks a bunch Justin, this helps us out a lot. I'll start looking into the switch.

Thanks again,

Chris

0 Richard Jiang over 14 years ago in reply to Chris Thunes

TI__Prodigy 320 points

Hello Justin,

I have trouble running external loopback test with PCIe/SGMII/SRIO AMC Breakout Card. The PDK version is 1.0.0.12. I'm using the multicore loopback example. There's no problem when running internal loopback, all 4 cores passed. But for external loopback, only core#2 passed. Here's the sys log,

[C66xx_0] ****************************************************
[C66xx_0] ****** Multicore Loopback Testing (Core 0) *******
[C66xx_0] ****************************************************
[C66xx_1] ****************************************************
[C66xx_1] ****** Multicore Loopback Testing (Core 1) *******
[C66xx_1] ****************************************************
[C66xx_2] ****************************************************
[C66xx_2] ****** Multicore Loopback Testing (Core 2) *******
[C66xx_2] ****************************************************
[C66xx_3] ****************************************************
[C66xx_3] ****** Multicore Loopback Testing (Core 3) *******
[C66xx_3] ****************************************************
[C66xx_3] Debug(Core 3): Waiting for SRIO to be initialized.
[C66xx_0] Debug(Core 0): Waiting for SRIO to be initialized.
[C66xx_1] Debug(Core 1): System Initialization for CPPI & QMSS
[C66xx_2] Debug(Core 2): Waiting for SRIO to be initialized.
[C66xx_1] Debug: Queue Manager and CPPI are initialized.
[C66xx_1] Debug: Host Region 0x83dc30
[C66xx_1] Debug(Core 1): SRIO Driver has been initialized
[C66xx_1] Debug: SRIO Driver has been started Instance Handle 0x@008312e0
[C66xx_1] Debug: SRIO Driver ISR has been registered
[C66xx_1] **********************************************
[C66xx_1] ******** Multicore Testing (Core 1) *********
[C66xx_1] **********************************************
[C66xx_0] Debug(Core 0): SRIO can now be used.
[C66xx_2] Debug(Core 2): SRIO can now be used.
[C66xx_3] Debug(Core 3): SRIO can now be used.
[C66xx_0] Debug: SRIO Driver has been started Instance Handle 0x@008312e0
[C66xx_2] Debug: SRIO Driver has been started Instance Handle 0x@008312e0
[C66xx_3] Debug: SRIO Driver has been started Instance Handle 0x@008312e0
[C66xx_0] Debug: SRIO Driver ISR has been registered
[C66xx_2] Debug: SRIO Driver ISR has been registered
[C66xx_3] Debug: SRIO Driver ISR has been registered
[C66xx_0] **********************************************
[C66xx_2] **********************************************
[C66xx_3] **********************************************
[C66xx_0] ******** Multicore Testing (Core 0) *********
[C66xx_2] ******** Multicore Testing (Core 2) *********
[C66xx_3] ******** Multicore Testing (Core 3) *********
[C66xx_0] **********************************************
[C66xx_2] **********************************************
[C66xx_3] **********************************************
[C66xx_3] ------------------------------------------------------
[C66xx_3] Debug(Core 3): Waiting for data to arrive 0x5678
[C66xx_0] ------------------------------------------------------
[C66xx_1] ------------------------------------------------------
[C66xx_2] ------------------------------------------------------
[C66xx_0] Debug(Core 0): Waiting for data to arrive 0xbeef
[C66xx_1] Debug(Core 1): Successfully sent data to ID:0x1234
[C66xx_2] Debug(Core 2): Waiting for data to arrive 0x1234
[C66xx_1] Debug(Core 1): Waiting for data to arrive 0x4560
[C66xx_2] Debug(Core 2): Successfully received 104 bytes
[C66xx_2] Debug(Core 2): Successfully sent data to ID:0x5678
[C66xx_2] Debug(Core 2): Multicore Test Passed
[C66xx_2] Debug(Core 2): Allocation Counter : 16
[C66xx_2] Debug(Core 2): Free Counter : 7
[C66xx_2] Debug(Core 2): Multicore Example successful.

And please see the modified device_srio_loopback.c.

Fullscreen 5102.device_srio_loopback.c.txt Download

/**
 *   @file  device_srio.c
 *
 *   @brief   
 *      The 6608 SRIO Device specific code. The SRIO driver calls out
 *      this code to initialize the SRIO IP block. The file is provided as 
 *      a sample configuration and should be modified by customers for 
 *      their own platforms and configurations.
 *
 *  \par
 *  NOTE:
 *      (C) Copyright 2010 Texas Instruments, Inc.
 * 
 *  Redistribution and use in source and binary forms, with or without 
 *  modification, are permitted provided that the following conditions 
 *  are met:
 *
 *    Redistributions of source code must retain the above copyright 
 *    notice, this list of conditions and the following disclaimer.
 *
 *    Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the 
 *    documentation and/or other materials provided with the   
 *    distribution.
 *
 *    Neither the name of Texas Instruments Incorporated nor the names of
 *    its contributors may be used to endorse or promote products derived
 *    from this software without specific prior written permission.
 *
 *  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 
 *  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 
 *  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 *  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 
 *  OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 
 *  SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 
 *  LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 *  DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 *  THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 
 *  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 
 *  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 *
 *  \par
*/

/* SRIO Driver Includes. */
#include <ti/drv/srio/srio_types.h>
#include <ti/drv/srio/include/listlib.h>
#include <ti/drv/srio/srio_drv.h>

/* CSL SRIO Functional Layer */
#include <ti/csl/csl_srio.h>
#include <ti/csl/csl_srioAux.h>
#include <ti/csl/csl_srioAuxPhyLayer.h>

/* CSL BootCfg Module */
#include <ti/csl/csl_bootcfg.h>
#include <ti/csl/csl_bootcfgAux.h>

/* CSL Chip Functional Layer */
#include <ti/csl/csl_chip.h>

/* QMSS Include */
#include <ti/drv/qmss/qmss_drv.h>

//#define LOOPBACK

/**********************************************************************
 ************************* LOCAL Definitions **************************
 **********************************************************************/

/* These are the GARBAGE queues which are used by the TXU to dump the 
 * descriptor if there is an error instead of recycling the descriptor
 * to the free queue. */
#define GARBAGE_LEN_QUEUE		    905
#define GARBAGE_TOUT_QUEUE		    906
#define GARBAGE_RETRY_QUEUE		    907
#define GARBAGE_TRANS_ERR_QUEUE	    908
#define GARBAGE_PROG_QUEUE		    909
#define GARBAGE_SSIZE_QUEUE		    910

/* SRIO Device Information
 * - 16 bit Device Identifier.
 * - 8 bit Device Identifier.
 * - Vendor Identifier. 
 * - Device Revision. */
#define DEVICE_VENDOR_ID            0x30
#define DEVICE_REVISION             0x0

/* SRIO Assembly Information
 * - Assembly Identifier
 * - Assembly Vendor Identifier. 
 * - Assembly Device Revision. 
 * - Assembly Extension Features */
#define DEVICE_ASSEMBLY_ID          0x0
#define DEVICE_ASSEMBLY_VENDOR_ID   0x30
#define DEVICE_ASSEMBLY_REVISION    0x0
#define DEVICE_ASSEMBLY_INFO        0x0100

/**********************************************************************
 ************************* Extern Definitions *************************
 **********************************************************************/

extern const uint32_t DEVICE_ID1_16BIT;
extern const uint32_t DEVICE_ID1_8BIT;
extern const uint32_t DEVICE_ID2_16BIT;
extern const uint32_t DEVICE_ID2_8BIT;
extern const uint32_t DEVICE_ID3_16BIT;
extern const uint32_t DEVICE_ID3_8BIT_ID;
extern const uint32_t DEVICE_ID4_16BIT;
extern const uint32_t DEVICE_ID4_8BIT_ID;

/**********************************************************************
 *********************** DEVICE SRIO FUNCTIONS ***********************
 **********************************************************************/

/** @addtogroup SRIO_DEVICE_API
 @{ */

/**
 *  @b Description
 *  @n  
 *      The function provides the initialization sequence for the SRIO IP
 *      block. This can be modified by customers for their application and
 *      configuration.
 *
 *  @retval
 *      Success     -   0
 *  @retval
 *      Error       -   <0
 */
int32_t SrioDevice_init (void)
{
    CSL_SrioHandle      hSrio;
	int32_t             i;
    SRIO_PE_FEATURES    peFeatures;
    SRIO_OP_CAR         opCar;
    Qmss_QueueHnd       queueHnd;
    uint8_t             isAllocated;
    uint32_t            gargbageQueue[] = { GARBAGE_LEN_QUEUE,  GARBAGE_TOUT_QUEUE,
                                            GARBAGE_RETRY_QUEUE,GARBAGE_TRANS_ERR_QUEUE,
                                            GARBAGE_PROG_QUEUE, GARBAGE_SSIZE_QUEUE };

    /* Get the CSL SRIO Handle. */
    hSrio = CSL_SRIO_Open (0);
    if (hSrio == NULL)
        return -1;
 
    /* Disable the SRIO Global block */
   	CSL_SRIO_GlobalDisable (hSrio);
   	
   	/* Disable each of the individual SRIO blocks. */
   	for(i = 0; i <= 9; i++)
   		CSL_SRIO_DisableBlock(hSrio, i);

    /* Set boot complete to be 0; we are not done with the initialization. */	
	CSL_SRIO_SetBootComplete(hSrio, 0);

    /* Now enable the SRIO block and all the individual blocks also. */
    CSL_SRIO_GlobalEnable (hSrio);
    for(i = 0; i <= 9; i++)
        CSL_SRIO_EnableBlock(hSrio,i);

    /* Configure SRIO ports to operate in loopback mode. */
#ifdef LOOPBACK
    CSL_SRIO_SetLoopbackMode(hSrio, 0);
    CSL_SRIO_SetLoopbackMode(hSrio, 1);
    CSL_SRIO_SetLoopbackMode(hSrio, 2);
    CSL_SRIO_SetLoopbackMode(hSrio, 3);
#else
    CSL_SRIO_SetNormalMode(hSrio, 0);
    CSL_SRIO_SetNormalMode(hSrio, 1);
    CSL_SRIO_SetNormalMode(hSrio, 2);
    CSL_SRIO_SetNormalMode(hSrio, 3);
#endif
	/* Enable Automatic Priority Promotion of response packets. */
	CSL_SRIO_EnableAutomaticPriorityPromotion(hSrio);

	/* Set the SRIO Prescalar select to operate in the range of 44.7 to 89.5 */
	CSL_SRIO_SetPrescalarSelect (hSrio, 0);

    /* Unlock the Boot Configuration Kicker */
    CSL_BootCfgUnlockKicker ();

    /* Assuming the link rate is 5000; program the PLL accordingly. */
    CSL_BootCfgSetSRIOSERDESConfigPLL (0x235);

#ifdef LOOPBACK
    /* Configure the SRIO SERDES Receive Configuration. */
    CSL_BootCfgSetSRIOSERDESRxConfig (0, 0x01C40495);
    CSL_BootCfgSetSRIOSERDESRxConfig (1, 0x01C40495);
    CSL_BootCfgSetSRIOSERDESRxConfig (2, 0x01C40495);
    CSL_BootCfgSetSRIOSERDESRxConfig (3, 0x01C40495);

    /* Configure the SRIO SERDES Transmit Configuration. */
    CSL_BootCfgSetSRIOSERDESTxConfig (0, 0x00780795);
    CSL_BootCfgSetSRIOSERDESTxConfig (1, 0x00780795);
    CSL_BootCfgSetSRIOSERDESTxConfig (2, 0x00780795);
    CSL_BootCfgSetSRIOSERDESTxConfig (3, 0x00780795);
#else
    /* Configure the SRIO SERDES Receive Configuration. */
    CSL_BootCfgSetSRIOSERDESRxConfig (0, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (1, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (2, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (3, 0x00440495);

    /* Configure the SRIO SERDES Transmit Configuration. */
    CSL_BootCfgSetSRIOSERDESTxConfig (0, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (1, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (2, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (3, 0x00180795);
#endif

#ifndef SIMULATOR_SUPPORT
    /* Loop around till the SERDES PLL is not locked. */
    while (1)
    {
        uint32_t    status;

        /* Get the SRIO SERDES Status */
        CSL_BootCfgGetSRIOSERDESStatus(&status);
        if (status & 0x1)
            break;
    }
#endif

    /* Clear the LSU pending interrupts. */
    CSL_SRIO_ClearLSUPendingInterrupt (hSrio, 0xFFFFFFFF, 0xFFFFFFFF);

    /* Set the Device Information */
    CSL_SRIO_SetDeviceInfo (hSrio, DEVICE_ID1_16BIT, DEVICE_VENDOR_ID, DEVICE_REVISION);

    /* Set the Assembly Information */
    CSL_SRIO_SetAssemblyInfo(hSrio, DEVICE_ASSEMBLY_ID, DEVICE_ASSEMBLY_VENDOR_ID, 
                             DEVICE_ASSEMBLY_REVISION, DEVICE_ASSEMBLY_INFO);

    /* TODO: Configure the processing element features
     *  The SRIO RL file is missing the Re-transmit Suppression Support (Bit6) field definition */
    peFeatures.isBridge                          = 0;
    peFeatures.isEndpoint                        = 0;
    peFeatures.isProcessor                       = 1;
    peFeatures.isSwitch                          = 0;
    peFeatures.isMultiport                       = 0;
    peFeatures.isFlowArbiterationSupported       = 0;
    peFeatures.isMulticastSupported              = 0;
    peFeatures.isExtendedRouteConfigSupported    = 0;
    peFeatures.isStandardRouteConfigSupported    = 1;
    peFeatures.isFlowControlSupported            = 1;
    peFeatures.isCRFSupported                    = 0;
    peFeatures.isCTLSSupported                   = 1;
    peFeatures.isExtendedFeaturePtrValid         = 1;
    peFeatures.numAddressBitSupported            = 1;
    CSL_SRIO_SetProcessingElementFeatures (hSrio, &peFeatures);

    /* Configure the source operation CAR */
    memset ((void *) &opCar, 0, sizeof (opCar));
    opCar.portWriteOperationSupport = 1;
    opCar.atomicClearSupport        = 1;
    opCar.atomicSetSupport          = 1;
    opCar.atomicDecSupport          = 1;
    opCar.atomicIncSupport          = 1;
    opCar.atomicTestSwapSupport     = 1;
    opCar.doorbellSupport           = 1;
    opCar.dataMessageSupport        = 1;
    opCar.writeResponseSupport      = 1;
    opCar.streamWriteSupport        = 1;
    opCar.writeSupport              = 1;
    opCar.readSupport               = 1;
    opCar.dataStreamingSupport      = 1;
    CSL_SRIO_SetSourceOperationCAR (hSrio, &opCar);

    /* Configure the destination operation CAR */
	memset ((void *) &opCar, 0, sizeof (opCar));
    opCar.portWriteOperationSupport  = 1;
    opCar.doorbellSupport            = 1;
    opCar.dataMessageSupport         = 1;
    opCar.writeResponseSupport       = 1;
    opCar.streamWriteSupport         = 1;
    opCar.writeSupport               = 1;
    opCar.readSupport                = 1;
    CSL_SRIO_SetDestOperationCAR (hSrio, &opCar);

    /* Set the 16 bit and 8 bit identifier for the SRIO Device. */
    CSL_SRIO_SetDeviceIDCSR (hSrio, DEVICE_ID1_8BIT, DEVICE_ID1_16BIT);

    /* Enable TLM Base Routing Information for Maintainance Requests & ensure that
     * the BRR's can be used by all the ports. */
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 0, 1, 1, 1, 0);
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 0, 2, 1, 1, 0);
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 0, 3, 1, 1, 0);
    CSL_SRIO_SetTLMPortBaseRoutingInfo(hSrio, 1, 0, 1, 1, 0);

    /* Configure the Base Routing Register to ensure that all packets matching the 
     * Device Identifier & the Secondary Device Id are admitted. */
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 0, 1, DEVICE_ID2_16BIT, 0xFFFF);
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 0, 2, DEVICE_ID3_16BIT, 0xFFFF);
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 0, 3, DEVICE_ID4_16BIT, 0xFFFF);
    CSL_SRIO_SetTLMPortBaseRoutingPatternMatch(hSrio, 1, 0, DEVICE_ID2_8BIT,  0xFF);

    /* We need to open the Garbage collection queues in the QMSS. This is done to ensure that 
     * these queues are not opened by another system entity. */
    for (i = 0; i < 6; i++)
    {
        /* Open the Garabage queues */
        queueHnd = Qmss_queueOpen (Qmss_QueueType_GENERAL_PURPOSE_QUEUE, gargbageQueue[i], &isAllocated);
        if (queueHnd < 0)
            return -1;

        /* Make sure the queue has not been opened already; we dont the queues to be shared by some other
         * entity in the system. */
        if (isAllocated > 1)
            return -1;
    }

    /* Set the Transmit Garbage Collection Information. */
    CSL_SRIO_SetTxGarbageCollectionInfo (hSrio, GARBAGE_LEN_QUEUE, GARBAGE_TOUT_QUEUE, 
                                         GARBAGE_RETRY_QUEUE, GARBAGE_TRANS_ERR_QUEUE, 
                                         GARBAGE_PROG_QUEUE, GARBAGE_SSIZE_QUEUE);

    /* Set the Host Device Identifier. */
    CSL_SRIO_SetHostDeviceID (hSrio, DEVICE_ID1_16BIT);

    /* Configure the component tag CSR */
    CSL_SRIO_SetCompTagCSR (hSrio, 0x00000000);

    /* Configure the PLM for all the ports. */
	for (i = 0; i < 4; i++)
	{
	    /* Set the PLM Port Silence Timer. */	
        CSL_SRIO_SetPLMPortSilenceTimer (hSrio, i, 0x2);

        /* TODO: We need to ensure that the Port 0 is configured to support both
         * the 2x and 4x modes. The Port Width field is read only. So here we simply
         * ensure that the Input and Output ports are enabled. */
        CSL_SRIO_EnableInputPort (hSrio, i);
        CSL_SRIO_EnableOutputPort (hSrio, i);

        /* Set the PLM Port Discovery Timer. */
        CSL_SRIO_SetPLMPortDiscoveryTimer (hSrio, i, 0x2);

        /* Reset the Port Write Reception capture. */
        CSL_SRIO_SetPortWriteReceptionCapture(hSrio, i, 0x0);
    }

    /* Set the Port link timeout CSR */
    CSL_SRIO_SetPortLinkTimeoutCSR (hSrio, 0x000FFF);

    /* Set the Port General CSR: Only executing as Master Enable */
    CSL_SRIO_SetPortGeneralCSR (hSrio, 0, 1, 0);

    /* Clear the sticky register bits. */
    CSL_SRIO_SetLLMResetControl (hSrio, 1);

    /* Set the device id to be 0 for the Maintenance Port-Write operation 
     * to report errors to a system host. */
    CSL_SRIO_SetPortWriteDeviceId (hSrio, 0x0, 0x0, 0x0);

    /* Set the Data Streaming MTU */
    CSL_SRIO_SetDataStreamingMTU (hSrio, 64);

    /* Configure the path mode for the ports. */
    for(i = 0; i < 4; i++)
        CSL_SRIO_SetPLMPortPathControlMode (hSrio, i, 0);

    /* Set the LLM Port IP Prescalar. */
    CSL_SRIO_SetLLMPortIPPrescalar (hSrio, 0x21);

    /* Enable the peripheral. */
    CSL_SRIO_EnablePeripheral(hSrio);

    /* Configuration has been completed. */
    CSL_SRIO_SetBootComplete(hSrio, 1);

#ifndef SIMULATOR_SUPPORT
    /* This code checks if the ports are operational or not. The functionality is not supported 
     * on the simulator. */    
	for(i = 0; i < 4; i++)
        while (CSL_SRIO_IsPortOk (hSrio, i) != TRUE);
#endif

    /* Set all the queues 0 to operate at the same priority level and to send packets onto Port 0 */
    for (i =0 ; i < 16; i++)
        CSL_SRIO_SetTxQueueSchedInfo(hSrio, i, 0, 0);

    /* Set the Doorbell route to determine which routing table is to be used 
     * This configuration implies that the Interrupt Routing Table is configured as 
     * follows:-
     *  Interrupt Destination 0 - INTDST 16 
     *  Interrupt Destination 1 - INTDST 17 
     *  Interrupt Destination 2 - INTDST 18
     *  Interrupt Destination 3 - INTDST 19 
     */
    CSL_SRIO_SetDoorbellRoute(hSrio, 0);

    /* Route the Doorbell interrupts. 
     *  Doorbell Register 0 - All 16 Doorbits are routed to Interrupt Destination 0. 
     *  Doorbell Register 1 - All 16 Doorbits are routed to Interrupt Destination 1. 
     *  Doorbell Register 2 - All 16 Doorbits are routed to Interrupt Destination 2. 
     *  Doorbell Register 3 - All 16 Doorbits are routed to Interrupt Destination 3. */
    for (i = 0; i < 16; i++)
    {
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 0, i, 0);
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 1, i, 1);
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 2, i, 2);
        CSL_SRIO_RouteDoorbellInterrupts(hSrio, 3, i, 3);
    }

    /* Initialization has been completed. */
    return 0;
}

/**
@}
*/

Richard

0 tscheck over 14 years ago in reply to Richard Jiang

TI__Mastermind 23525 points

Richard,

The pll multiplier you are using is illegal. There are multple threads on this, but please see this one to change it accordingly.

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/114940/464064.aspx#464064

Regards,

Travis

0 Richard Jiang over 14 years ago in reply to tscheck

TI__Prodigy 320 points

Travis,

I tried all PLL settings in the link, and no help. It is the wire scheme making difference.

I found in the example all txqueues are hardcoded using port0. Then I changed code to make core0 use port0, core1 use port1, core2 use port2, core3 use port3. And then I wired as port0->port1->port2->port3->port0, the test only passed on core2. When I rewired as port0<->port1, port2<->port3, all tests passed. All tests also got passed when wired as, port0<->port1, port2<->port2, port3<->port3.

Thanks,

Richard

0 tscheck over 14 years ago in reply to Richard Jiang

TI__Mastermind 23525 points

Richard,

SRIO mandates that both the TX and RX pair be routed to the same device. This is because there are physical layer handshaking going on that requires direct communication between link partners. So if you want to do loopback testing, you can use cables to loopback any port's TX pair to the same port's RX pair, or you can directly link any two ports together such that TX-->RX in both directions.

Richard Jiang said:
And then I wired as port0->port1->port2->port3->port0, the test only passed on core2.

There is no way this can pass data.

Richard Jiang said:
I tried all PLL settings in the link, and no help. It is the wire scheme making difference.

The other part of this thread shows that you have to change the Serdes CFGTX and CFGRX too. The default settings for these also put the Serdes in loopback. These lines immediately below control the digitial loopback which happens before the SerDes.

    /* Configure SRIO to operate in loopback mode. */
//    CSL_SRIO_SetLoopbackMode(hSrio,0);
//    CSL_SRIO_SetLoopbackMode(hSrio,1);
//    CSL_SRIO_SetLoopbackMode(hSrio,2);
//    CSL_SRIO_SetLoopbackMode(hSrio,3);

    /* Configure SRIO to operate in normal mode. */
    CSL_SRIO_SetNormalMode(hSrio,0);
    CSL_SRIO_SetNormalMode(hSrio,1);
    CSL_SRIO_SetNormalMode(hSrio,2);
    CSL_SRIO_SetNormalMode(hSrio,3);

There are also loopback capabilities inside the serdes and they are controlled as below:

    /* Configure the SRIO SERDES Receive Configuration. */
    /* loopback mode */
// CSL_BootCfgSetSRIOSERDESRxConfig (0, 0x01C40495);
// CSL_BootCfgSetSRIOSERDESRxConfig (1, 0x01C40495);
// CSL_BootCfgSetSRIOSERDESRxConfig (2, 0x01C40495);
// CSL_BootCfgSetSRIOSERDESRxConfig (3, 0x01C40495);

    /* Non-loopback */
    CSL_BootCfgSetSRIOSERDESRxConfig (0, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (1, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (2, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (3, 0x00440495);

    /* Configure the SRIO SERDES Transmit Configuration. */
    /* loopback mode */
// CSL_BootCfgSetSRIOSERDESTxConfig (0, 0x00780795);
// CSL_BootCfgSetSRIOSERDESTxConfig (1, 0x00780795);
// CSL_BootCfgSetSRIOSERDESTxConfig (2, 0x00780795);
// CSL_BootCfgSetSRIOSERDESTxConfig (3, 0x00780795);

    /* Non-loopback */
    CSL_BootCfgSetSRIOSERDESTxConfig (0, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (1, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (2, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (3, 0x00180795);

So if you want the C6678EVM (312.5Mhz refclk) to be enabled for 3.125Gbps data for example (no internal loopback), you need to change:

CSL_BootCfgSetSRIOSERDESConfigPLL (0x229);

    CSL_BootCfgSetSRIOSERDESTxConfig (0, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (1, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (2, 0x00180795);
    CSL_BootCfgSetSRIOSERDESTxConfig (3, 0x00180795);

    CSL_BootCfgSetSRIOSERDESRxConfig (0, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (1, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (2, 0x00440495);
    CSL_BootCfgSetSRIOSERDESRxConfig (3, 0x00440495);

   /* Configure SRIO to operate in normal mode. */
    CSL_SRIO_SetNormalMode(hSrio,0);
    CSL_SRIO_SetNormalMode(hSrio,1);
    CSL_SRIO_SetNormalMode(hSrio,2);
    CSL_SRIO_SetNormalMode(hSrio,3);

Regards,

Travis

0 Richard Jiang over 14 years ago in reply to tscheck

TI__Prodigy 320 points

Travis,

Thanks a lot. Now I understand why the long loop test does not work.

Richard

Processors

Processors forum

C6678 SRIO External Loopback