This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Making SRIO and NDK work together

Hi,

I am having problem making SRIO and NDK work together. Basically I added parts from NDK example to SRIO one and made sure that network works (udp echo and ping). Network and SRIO work separately but if I try to use them together one of the drivers fails.

If I initialize network and then start SRIO driver I get "Error: Program Accumulator failed". If I start SRIO first then it works until network task starts. I tried changing srio accumulator channel number to 1, event combiner to 49, queue number and other srio driver parameters in different combinations but without any success.

It seems my QMSS/CPPI/PA configuration is not right but I don't get which part is at fault. Is there any example project or configuration of working srio and ndk? Should I try to configure SRIO to use PA (documentation is too sparse on how to do this)?

I am using c6678 evaluation board, NDK 2.21.0.31, MCSDK 2.0.9.21

Thank you,

Alexey

  • Here's my init code (mostly).  I tried to highlight what I remember changing to make NDK and SRIO work together.  I know I had to change the event number, the queues and the memory region.  I also probably have other posts on the forum that might help.

    good luck!

     

    Int32 system_init (Void)

    {

        Int32               result;

        Qmss_MemRegInfo     memRegInfo;

        QMSS_CFG_T          qmss_cfg;

        CPPI_CFG_T          cppi_cfg;

            char                           buffer[MAX_MESSAGE_SIZE];

     

        /* Initialize the QMSS Configuration block. */

        memset (&qmssInitConfig, 0, sizeof (Qmss_InitCfg));

     

        /* Initialize the Host Region. */

        memset ((void *)&host_region, 0, sizeof(host_region));

     

                    if (platform_get_coreid() == 0)

                    {

                            qmss_cfg.master_core        = 1;

                    }

                    else

                    {

                            qmss_cfg.master_core        = 0;

                    }

            qmss_cfg.max_num_desc       = MAX_NUM_DESC + NUM_HOST_DESC;

            qmss_cfg.num_nimu_desc          = MAX_NUM_DESC;

            qmss_cfg.desc_size          = MAX_DESC_SIZE;

            qmss_cfg.mem_region         = Qmss_MemRegion_MEMORY_REGION0;

     

            if (res_mgr_init_qmss (&qmss_cfg) != ITS_A_OKAY)

            {

                return PROCESS_CALL_FAILED;

            }

            else

            {

                    LOG_INFO("QMSS successfully initialized. ");

            }

     

                    /* Memory Region 0 Configuration */

                    memRegInfo.descBase         = (UInt32 *)l2_global_address((UInt32)host_region);

                    memRegInfo.descSize         = SIZE_HOST_DESC;

                    memRegInfo.descNum          = NUM_HOST_DESC;

                    memRegInfo.manageDescFlag   = Qmss_ManageDesc_MANAGE_DESCRIPTOR;

                    memRegInfo.memRegion        = Qmss_MemRegion_MEMORY_REGION1;

                    memRegInfo.startIndex       = 128;//  0;

     

                    /* Initialize and insert the memory region. */

                    result = Qmss_insertMemoryRegion (&memRegInfo);

                    if (result < QMSS_SOK)

                    {

                            sprintf(buffer, "Problem inserting memory region: %d", result);

                            ekg_vfib(buffer, CORE0_INIT_TASK, CORE0_INIT_TASK);

                            return PROCESS_CALL_FAILED;

                    }

     

            /* Initialize CPPI */

            if (platform_get_coreid() == 0)

            {

                    cppi_cfg.master_core        = 1;

            }

            else

            {

                    cppi_cfg.master_core        = 0;

            }

            cppi_cfg.dma_num            = Cppi_CpDma_PASS_CPDMA;

            cppi_cfg.num_tx_queues      = NUM_PA_TX_QUEUES;

            cppi_cfg.num_rx_channels    = NUM_PA_RX_CHANNELS;

            if (res_mgr_init_cppi (&cppi_cfg) != ITS_A_OKAY)

            {

               sprintf(buffer, "Failed to initialize CPPI subsystem ");

               ekg_vfib(buffer, CORE0_INIT_TASK, CORE0_INIT_TASK);

               return PROCESS_CALL_FAILED;

            }

            else

            {

               LOG_INFO("CPPI successfully initialized ");

            }

     

            if (res_mgr_init_pass()!= ITS_A_OKAY)

            {

                sprintf(buffer, "Failed to initialize Packet Accelerator ");

                ekg_vfib(buffer, CORE0_INIT_TASK, CORE0_INIT_TASK);

                    return PROCESS_CALL_FAILED;

            }

            else

            {

                     LOG_INFO("Packet Accelerator successfully initialized ");

            }

     

        /* CPPI and Queue Manager are initialized. */

        LOG_INFO ("Queue Manager, Packet Accelerator and CPPI are initialized.");

        return ITS_A_OKAY;

    }

     

    Srio_DrvHandle initalizeSRIODriver(void)

    {

        Qmss_QueueHnd   myRxFreeQueueHnd;

        Qmss_QueueHnd   myRxCompletionQueueHnd;

        Qmss_QueueHnd   tmpQueueHnd;

        UInt32          numAllocated;

        int32_t         eventId;

        UInt8           isAllocated;

        Cppi_DescCfg    descCfg;

        UInt16          index;

        Cppi_HostDesc*  ptrHostDesc;

        UInt8*          ptrRxData;

        UInt32          numRxBuffers;

        Srio_DrvConfig  drvCfg;

        Srio_DrvHandle  drvHandle;

        CSL_SrioHandle  hSrioCSL;

            char                   buffer[MAX_MESSAGE_SIZE];

     

            /* Initialize the SRIO Driver Configuration. */

            memset ((Void *)&drvCfg, 0, sizeof(Srio_DrvConfig));

     

            /* Initialize the OSAL

             * Gets 30 buffers of mtu size. 30 is hard coded in the OSAL.

             * */

            if (Osal_dataBufferInitMemory(SRIO_MAX_MTU) < 0)

            {

                    return NULL;

            }

     

            /********************************************************************************

             * The SRIO Driver Instance is going to be created with the following properties:

             * - Application Managed

             * - Receive Completion Queue is Application specified; which implies that there

             *   is no interrupt support. Applications will hence need to poll the queue to

             *   check if there is data available or not.

             * - The Receive Free Descriptor Queues along with the Size thresholds are

             *   managed and created by the application.

             ********************************************************************************/

     

            /* Create the application receive free queue. */

            myRxFreeQueueHnd = Qmss_queueOpen (Qmss_QueueType_GENERAL_PURPOSE_QUEUE, QMSS_PARAM_NOT_SPECIFIED,

                                                                               &isAllocated);

            if (myRxFreeQueueHnd < 0)

            {

                    return NULL;

            }

     

            /* Create the application receive completion queue. */

            myRxCompletionQueueHnd = Qmss_queueOpen (Qmss_QueueType_GENERAL_PURPOSE_QUEUE, QMSS_PARAM_NOT_SPECIFIED,

                                                                                             &isAllocated);

            if (myRxCompletionQueueHnd < 0)

            {

              return NULL;

            }

     

            /* We are going to be using 4 receive buffers in this example. */

            numRxBuffers = 4;

     

            /* Application created queue which stores all the receive buffers. */

            descCfg.memRegion                 = Qmss_MemRegion_MEMORY_REGION1;//Qmss_MemRegion_MEMORY_REGION0; // aligns to Host region that was set in init, it was not specified which means it will be zero.

            descCfg.descNum                   = numRxBuffers;

            descCfg.destQueueNum              = QMSS_PARAM_NOT_SPECIFIED;

            descCfg.queueType                 = Qmss_QueueType_GENERAL_PURPOSE_QUEUE;

            descCfg.initDesc                  = Cppi_InitDesc_INIT_DESCRIPTOR;

            descCfg.descType                  = Cppi_DescType_HOST;

            descCfg.returnQueue               = Qmss_getQueueNumber(myRxFreeQueueHnd);

            descCfg.epibPresent               = Cppi_EPIB_NO_EPIB_PRESENT;

            descCfg.returnPushPolicy          = Qmss_Location_HEAD;

            descCfg.cfg.host.returnPolicy     = Cppi_ReturnPolicy_RETURN_ENTIRE_PACKET;

            descCfg.cfg.host.psLocation       = Cppi_PSLoc_PS_IN_DESC;

     

            tmpQueueHnd = Cppi_initDescriptor (&descCfg, &numAllocated);

            if (tmpQueueHnd < 0 || numAllocated < 1)

            {

            return NULL;

            }

     

            /* Initialize the application receive buffers. */

            for (index = 0; index < descCfg.descNum; index++)

            {

                    /* Pop off a descriptor */

                    ptrHostDesc = (Cppi_HostDesc *)Qmss_queuePop(tmpQueueHnd);

                    if (ptrHostDesc == NULL)

                    {

                            return NULL;

                    }

     

                    /* Allocate the receive buffer where the data will be received into by the SRIO CPDMA.

                     * osal_srio grabs one of the 30 buffers allocated by the init function. */

                    ptrRxData = (UInt8*)Osal_srioDataBufferMalloc(SRIO_MAX_MTU);

                    if (ptrRxData == NULL)

                    {

                            return NULL;

                    }

     

                    /* Set the DATA and ORIGNAL DATA in the buffer descriptor. */

                    Cppi_setData (Cppi_DescType_HOST, (Cppi_Desc*)ptrHostDesc, (UInt8*)ptrRxData, SRIO_MAX_MTU);

                    Cppi_setOriginalBufInfo (Cppi_DescType_HOST, (Cppi_Desc*)ptrHostDesc, (UInt8*)ptrRxData, SRIO_MAX_MTU);

     

                    /* Add the packet descriptor to the Application Receive Free Queue. */

                    Qmss_queuePushDescSize (myRxFreeQueueHnd, (UInt32*)ptrHostDesc, SIZE_HOST_DESC);

     

            }

     

            /* Close the temporary queue. */

            Qmss_queueClose (tmpQueueHnd);

     

            /********************************************************************************

             * The SRIO Driver Instance is going to be created with the following properties:

             * - Driver Managed

             * - Interrupt Support (Pass the Rx Completion Queue as NULL)

             ********************************************************************************/

     

            /* Setup the SRIO Driver Managed Configuration. */

            drvCfg.bAppManagedConfig = FALSE;

     

            /* Driver Managed: Receive Configuration */

            drvCfg.u.drvManagedCfg.bIsRxCfgValid             = 1;

            drvCfg.u.drvManagedCfg.rxCfg.rxMemRegion         = Qmss_MemRegion_MEMORY_REGION1; //Qmss_MemRegion_MEMORY_REGION0;

            drvCfg.u.drvManagedCfg.rxCfg.numRxBuffers        = 4;

            drvCfg.u.drvManagedCfg.rxCfg.rxMTU               = SRIO_MAX_MTU;

     

            /* Accumulator Configuration. */

            int32_t coreToQueueSelector[4];

     

      /* This is the table which maps the core to a specific receive queue. */

            //These were chosen based on using event 49 instead of 48.  NDK uses 48

            coreToQueueSelector[0] = 712;

            coreToQueueSelector[1] = 713;

            coreToQueueSelector[2] = 714;

            coreToQueueSelector[3] = 715;

     

            /* Since we are programming the accumulator we want this queue to be a HIGH PRIORITY Queue */

            drvCfg.u.drvManagedCfg.rxCfg.rxCompletionQueue = Qmss_queueOpen (Qmss_QueueType_HIGH_PRIORITY_QUEUE,

                                                                                                                                             coreToQueueSelector[CORE_NUM], &isAllocated);

            if (drvCfg.u.drvManagedCfg.rxCfg.rxCompletionQueue < 0)

            {

                    return NULL;

            }

     

            /* Accumulator Configuration is VALID. */

            drvCfg.u.drvManagedCfg.rxCfg.bIsAccumlatorCfgValid = 1;

     

            /* Accumulator Configuration. */

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.channel             = NUM_PA_TX_QUEUES+NUM_PA_RX_CHANNELS+CORE_NUM;

     

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.command             = Qmss_AccCmd_ENABLE_CHANNEL;

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.queueEnMask         = 0;

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.queMgrIndex         = coreToQueueSelector[CORE_NUM];

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.maxPageEntries      = 2;

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.timerLoadCount      = 0;

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.interruptPacingMode = Qmss_AccPacingMode_LAST_INTERRUPT;

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.listEntrySize       = Qmss_AccEntrySize_REG_D;

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.listCountMode       = Qmss_AccCountMode_ENTRY_COUNT;

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.multiQueueMode      = Qmss_AccQueueMode_SINGLE_QUEUE;

     

            /* Initialize the accumulator list memory */

            memset ((Void *)&gHiPriAccumListSrio[0], 0, sizeof(gHiPriAccumListSrio));

            drvCfg.u.drvManagedCfg.rxCfg.accCfg.listAddress = l2_global_address((UInt32)&gHiPriAccumListSrio[0]);

     

            /* Driver Managed: Transmit Configuration */

            drvCfg.u.drvManagedCfg.bIsTxCfgValid             = 1;

            drvCfg.u.drvManagedCfg.txCfg.txMemRegion         = Qmss_MemRegion_MEMORY_REGION1;//Qmss_MemRegion_MEMORY_REGION0;

            drvCfg.u.drvManagedCfg.txCfg.numTxBuffers        = 4;

            drvCfg.u.drvManagedCfg.txCfg.txMTU               = SRIO_MAX_MTU;

     

            /* Start the Driver Managed SRIO Driver. Calls Cppi_open for SRIO */

            drvHandle = Srio_start(&drvCfg);

            if (drvHandle == NULL)

            {

                    return NULL;

            }

     

            //Hook up the SRIO interrupt with the core.  Don't use standard 48 becuase ndk is using that.

            EventCombiner_dispatchPlug (49, (EventCombiner_FuncPtr)Srio_rxCompletionIsr, (UArg)drvHandle, TRUE);

            EventCombiner_enableEvent(49);

     

            /* Get the CSL SRIO Handle. */

            hSrioCSL = CSL_SRIO_Open (0);

     

             /* SRIO DIO Interrupts need to be routed from the CPINTC0 to GEM Event.

             *  - We have configured DIO Interrupts to get routed to Interrupt Destination 0

             *    (Refer to the CSL_SRIO_RouteLSUInterrupts API configuration in the SRIO Initialization)

             *  - We want this System Interrupt to mapped to Host Interrupt 8 */

            /* Disable Interrupt Pacing for INTDST0 */

            CSL_SRIO_DisableInterruptPacing (hSrioCSL, 0);

            /* Route LSU0 ICR0 to INTDST0 */

            CSL_SRIO_RouteLSUInterrupts (hSrioCSL, 0, 0);

            /* Route LSU0 ICR1 to INTDST0 */

            CSL_SRIO_RouteLSUInterrupts (hSrioCSL, 1, 0);

            /* Route LSU0 ICR2 to INTDST0 */

            CSL_SRIO_RouteLSUInterrupts (hSrioCSL, 2, 0);

            /* Map the System Interrupt i.e. the Interrupt Destination 0 interrupt to the DIO ISR Handler. */

            CpIntc_dispatchPlug(CSL_INTC0_INTDST0, (CpIntc_FuncPtr)myDioTxCompletionIsr, (UArg)drvHandle, TRUE);

            /* The configuration is for CPINTC0. We map system interrupt 112 to Host Interrupt 8. */

            CpIntc_mapSysIntToHostInt(0, CSL_INTC0_INTDST0, 8);

            /* Enable the Host Interrupt. */

            CpIntc_enableHostInt(0, 8);

            /* Enable the System Interrupt */

            CpIntc_enableSysInt(0, CSL_INTC0_INTDST0);

            /* Get the event id associated with the host interrupt. */

            eventId = CpIntc_getEventId(8);

     

            LOG_DEBUG("Srio Event Id: %i attached to host interrupt 8", eventId);

     

            /* Plug the CPINTC Dispatcher. */

            EventCombiner_dispatchPlug (eventId, (EventCombiner_FuncPtr)CpIntc_dispatch, 8, TRUE);

     

            return drvHandle;

  • Hi,

    Thank you very much for the code. Unfortunately it did not help resolve my problem completely. Your code helped me figure out memory region configuration but for some reason SRIO does not work neither on accumulator channel != 0 or on event number 49 and queue number 712 (even without ndk driver initialization). It looks like packets are send away but nothing is received. If I use channel 0, event 48 and queue 704 I get data back.

    Do you have any idea why changing to another channel or event might block loopback? It seems you were using DIO I am testing everything using message passing, could this cause problems?

    Thank you,

    Alexey

    PS: I will try to make new project and use your code verbatim, maybe I missed something copying it.

  • Looks like I figured it out or at least made it work. It seems that accumulator channel number is somehow connected to event number and qmss queue number. The only combination that works is: accumulator channel 8, event 49, queue 712 (I use srio and ndk on core 0). If you try to use other accumulator channel with queue 712 then packets are not received. I tested this on  SRIO_MulticoreLoopbackexampleProject. Could some TI employee please explain this or give a link to documentation where this is mentioned?

    Another curious thing happens with Hwi_eventMap. Srio loopback example contains the following code:

    /* Map the event id to hardware interrupt 8. */
    Hwi_eventMap(8, 48);
    /* Enable interrupt 8. */
    Hwi_enableInterrupt(8);

    and if you look up documentation you can find the following example:

    // Maps GEM event #65 as the interrupt source of int vector #7

    Hwi_eventMap(7, 65);

    BUT if you check nimu_eth.c you will find the following pieces:

    #define PLATFORM_ETH_EVENTID        48        /**< Ethernet Switch event - Used by NIMU library */

    /* System event n - Accumulator Channel 0 */
        eventId         =   PLATFORM_ETH_EVENTID;

    /* Map the event id to hardware interrupt 7. */
        Hwi_eventMap(vectId, eventId >> 5);

    So do you have to devide the second argument by 32 or not? TI employee thinks that you have to divide by 32: http://e2e.ti.com/support/embedded/bios/f/355/t/144666.aspx


    If you add Hwi_eventMap(8, 49) then network stops working if you add Hwi_eventMap(8, 49>>5)  everything seems ok but it does not make sense. I just dropped this part of code and looks like it is working (probably srio is bound to interrupt 7 now).

    So to my understanding to make SRIO and NDK work you need:

    1. Set up memory regions more of less like in Brandy's example.

    2. Change to accumulator channel 8 (not NUM_PA_TX_QUEUES+NUM_PA_RX_CHANNELS+CORE_NUM), event 49 and qmss queue 712.

    3. Remove Hwi_eventMap and Hwi_enableInterrupt calls.

    Alexey

  • Alexey Naydenov said:
    So do you have to devide the second argument by 32 or not? TI employee thinks that you have to divide by 32: http://e2e.ti.com/support/embedded/bios/f/355/t/144666.aspx

    Can anyone finally clarify situation about shifting by 5 (dividing by 32) of EventId when calling Hwi_eventMap??? And why it is different in different MCSDK examples (NDK shifts by 5 while SrioMulticoreExample does not)? I am using the latest MCSDK 02.00.09 and they are different there.

    Ask again, please clarify this situation. 

  • It seems when calling to Hwi_eventMap we need to shift EventId to 5 (divide to 32). Now, with shifting, my SRIO code works fine on all (in my case, 7) cores.

    TI employees, if so, please, correct SrioMulticoreLoopback example in the next MCSDK (PDK) release.

  • Is there any example project for SRIO and NDK??? where can i find??

  • Hi Alexey Naydenov ,

    Above suggested point 2 - Change to accumulator channel 8 (not NUM_PA_TX_QUEUES+NUM_PA_RX_CHANNELS+CORE_NUM), event 49 and qmss queue 712

    helped alot and gave a break through for my application (Ethernet + SRIO), where we are debugging from 2 weeks.

    Thank you very much for your detailed inputs.

    Regards,
    Mani Kumar