This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

EDMA3+PCIe data space regions

Hi,

I’m using EDMA3 for PCIe high transfer performance [shannons]

Say that I have 2 or more EPs, and I want to perform a MWr from RC to all EPs at once with 1 EDMA transfer.

In EDMA config I see that the limit of DST_BIDX & DST_CIDX  is 2^16=64KB, while in PCIe data space the minimum is 1 MB/region !

there are ways to do this with a single transfer (both EPs receive data at the same time with a single triggering of EDMA3)?

  • Delared,

    I think you could always link multiple parameter sets to the same EDMA channel or chain multiple EDMA channels together for multiple transactions.

    Then you are free to use different src/dst setup in each parameter set and all of them could be linked by one EDMA triggering.

    But I am not sure how to understand the request of "same time". Are you trying to broadcast the same data to multiple EPs or are you trying to send different data buffers to different EPs but with one single triggering please?

  • Steven,

    For example, I have 4MB, and I want to transfer it from RC to EPs ( 1MB/EP).

    I’m using the EDMA0 like this:

                    EDMA_Transfer(0,1,0,srcBuf,pcieBase,pcie_max_payload,pcie_BCNT);

    /*Int32 EDMA_Transfer (Int32 instNum, Uint8 TCNum, Uint8 channelNum, uint32_t* src_data,uint32_t* dst_data,uint32_t ACNT,uint32_t BCNT)*/

                    /* instNum=0 : EDMA0

                                   TCNum=0 :Transfer controller 0 with DBS=128

                                   channelNum= 0: channel 0

                                   src_data=srcBuf: write from srcBuf (located in RC MSMC)

                                   dst_data=pcieBase: write to EP via PCIe memory space regions

                                   ACNT & BCNT: EDMA config

                    */

    So i can do it with 4 EDMA triggering :

    EDMA_Transfer(0,1,0,srcBuf,pcieBase,pcie_max_payload,pcie_BCNT); // to EP1

    EDMA_Transfer(0,1,0,srcBuf+offset_src1 ,pcieBase+ offsetEP2,pcie_max_payload,pcie_BCNT); // to EP2

    EDMA_Transfer(0,1,0,srcBuf+offset_src2 ,pcieBase+ offsetEP3 ,pcie_max_payload,pcie_BCNT); // to EP2

    EDMA_Transfer(0,1,0,srcBuf+offset_src3 ,pcieBase+ offsetEP4,pcie_max_payload,pcie_BCNT); // to EP4

     

    But, now I’m trying to replace this with a single EDMA triggering if it is possible!

    e.g New_EDMA_Transfer(new parameters); // to EP1/EP2/EP3/EP4

    or at least reduce the number of EDMA triggering by using the EDMA3 features (3D transfer, multi-TC, etc…)

     

    ---- i carry that my EPs will receive the data nearly at the same time, and I will not have a lot of time waiting (side EPs) so the 4Eps can  process the next task nearly at the same time  (depends on my application).

    how can i do this???

    and how can i change my EDMA function to supporte multi-TC/multi-channel...??

    /**********************************************************************
     ************************* EDMA Function*******************************
     **********************************************************************/
    Int32 EDMA_Transfer (Int32 instNum, Uint8 TCNum, Uint8 channelNum, uint32_t* src_data,uint32_t* dst_data,uint32_t ACNT,uint32_t BCNT)
    {
        CSL_Edma3Handle                 hModule;
        CSL_Edma3Obj                    edmaObj;
        CSL_Edma3ParamHandle            hParamPing;
        CSL_Edma3ChannelObj             chObj;
        CSL_Edma3CmdIntr                regionIntr;
        CSL_Edma3ChannelHandle          hChannel;
        CSL_Edma3ParamSetup             myParamSetup;
        CSL_Edma3Context                context;
        CSL_Edma3ChannelAttr            chAttr;
        CSL_Status                      status;




        /* Module initialization */
        if (CSL_edma3Init(&context) != CSL_SOK)
        {
            printf ("Error: EDMA module initialization failed\n");
            return -1;
        }

        /* Open the EDMA Module using the provided instance number */
        hModule = CSL_edma3Open(&edmaObj, instNum, NULL, &status);
        if ( (hModule == NULL) || (status != CSL_SOK))
        {
            printf ("Error: EDMA module open failed\n");
            return -1;
        }

        /* Channel open */
        chAttr.regionNum = CSL_EDMA3_REGION_GLOBAL;
        chAttr.chaNum    = channelNum;
        hChannel = CSL_edma3ChannelOpen(&chObj, instNum, &chAttr, &status);
        if ((hChannel == NULL) || (status != CSL_SOK))
        {
            printf ("Error: Unable to open EDMA Channel:%d\n", channelNum);
            return -1;
        }

        if(!instNum)
        {
            /* For first EDMA instance there are only 2 TCs and 2 event queues
             * Modify the channel default queue setup from 0 to 1
             */
             if (CSL_edma3HwChannelSetupQue(hChannel,TCNum) != CSL_SOK)
            {
                printf ("Error: EDMA channel setup queue failed\n");
                return -1;
            }
        }
        else
        {
            /* For EDMA instance 1 and 2 maximum of 4 TCs and 4 event queues are supported
             * Change Channel Default queue setup from 0 to 3
             * CSL_EDMA3_QUE_=TCNum
             */
            if (CSL_edma3HwChannelSetupQue(hChannel,TCNum) != CSL_SOK)
            {
                printf ("Error: EDMA channel setup queue failed\n");
                return -1;
            }
        }

        /* Map the DMA Channel to PARAM Block 2. */
        CSL_edma3MapDMAChannelToParamBlock (hModule, channelNum, 2);

        /* Obtain a handle to parameter set 2 */
        hParamPing = CSL_edma3GetParamHandle(hChannel, 2, &status);
        if (hParamPing == NULL)
        {
            printf ("Error: EDMA Get Parameter Entry failed for 2.\n");
            return -1;
        }

        /* Setup the parameter entry parameters (Ping buffer) */
        myParamSetup.option = CSL_EDMA3_OPT_MAKE(CSL_EDMA3_ITCCH_DIS, \
                                                 CSL_EDMA3_TCCH_DIS, \
                                                 CSL_EDMA3_ITCINT_DIS, \
                                                 CSL_EDMA3_TCINT_EN, \
                                                 3, CSL_EDMA3_TCC_NORMAL,\
                                                 CSL_EDMA3_FIFOWIDTH_NONE, \
                                                 CSL_EDMA3_STATIC_DIS, \
                                                 CSL_EDMA3_SYNC_AB, \
                                                 CSL_EDMA3_ADDRMODE_INCR, \
                                                 CSL_EDMA3_ADDRMODE_INCR );
        myParamSetup.srcAddr    = (Uint32)src_data;
        myParamSetup.aCntbCnt   = CSL_EDMA3_CNT_MAKE(ACNT,BCNT);
        myParamSetup.dstAddr    = (Uint32)dst_data;
        myParamSetup.srcDstBidx = CSL_EDMA3_BIDX_MAKE(SRC_BIDX,DST_BIDX);
        myParamSetup.linkBcntrld= CSL_EDMA3_LINKBCNTRLD_MAKE(0xFFFF,BCNT);
        myParamSetup.srcDstCidx = CSL_EDMA3_CIDX_MAKE(SRC_CIDX,DST_CIDX);
        myParamSetup.cCnt = CCNT;

        /* Ping setup */
        if (CSL_edma3ParamSetup(hParamPing,&myParamSetup) != CSL_SOK)
        {
            printf ("Error: EDMA Parameter Entry Setup failed\n");
            return -1;
        }

        /* Interrupt enable (Bits 0-1)  for the global region interrupts */
        regionIntr.region = CSL_EDMA3_REGION_GLOBAL;
        regionIntr.intr   = 0x0;
        regionIntr.intrh  = 0x0;
        CSL_edma3HwControl(hModule,CSL_EDMA3_CMD_INTR_ENABLE,&regionIntr);


        edma_before=_CSL_tscRead ();
        /* Trigger channel */
        CSL_edma3HwChannelControl(hChannel,CSL_EDMA3_CMD_CHANNEL_SET,NULL);


        /* Poll on IPR bit 0 */
        do {
            CSL_edma3GetHwStatus(hModule,CSL_EDMA3_QUERY_INTRPEND,&regionIntr);
        } while ((regionIntr.intr & 0x8) != 0x8);

         edma_after=_CSL_tscRead ();

        /* Clear the pending bit */
        CSL_edma3HwControl(hModule,CSL_EDMA3_CMD_INTRPEND_CLEAR,&regionIntr);

        /* Close channel */
        if (CSL_edma3ChannelClose(hChannel) != CSL_SOK)
        {
            printf("Error: EDMA Channel Close failed\n");
            return -1;
        }

        /* Close EDMA module */
        if (CSL_edma3Close(hModule) != CSL_SOK)
        {
            printf("Error: EDMA Module Close failed\n");
            return -1;
        }

        /* The test passed. */
        return 0;
    }

  • Delared,

    Please take a look at the section 2.3.7 Linking Transfers in EDMA user guide. It has the example of how to link multiple transfers together via multiple param sets (using different offsets for src/dst in the param sets).

    Please note that in the linking transfer, the #2 transfer will start after the #1 transfer completes since the 4 transactions are using the same EDMA TC/channel. So EP2 will wait for the 1MB transferred to EP1 first. 

    If you really need all 4 EPs to receive the data simultaneously, you probably need to use multiple TCs/channels. Then you have to setup 4 different EDMA handlers/channels and repeat the similar thing 4 times as you did for single transfer (using different param set with EP offset for different channel/TC). But you could pre-intialize everything ahead of time and trigger the 4 transfers together, such as:

        /* Trigger channel */
        CSL_edma3HwChannelControl(hChannel_1,CSL_EDMA3_CMD_CHANNEL_SET,NULL);

        CSL_edma3HwChannelControl(hChannel_2,CSL_EDMA3_CMD_CHANNEL_SET,NULL);

        CSL_edma3HwChannelControl(hChannel_3,CSL_EDMA3_CMD_CHANNEL_SET,NULL);

        CSL_edma3HwChannelControl(hChannel_4,CSL_EDMA3_CMD_CHANNEL_SET,NULL);

    And then you could check the IPR bit for each channel for the completion.

    For more EDMA usage, please refer to the EDMA user guide.

    Please keep in mind that we only have one PCIe port in the C6678 device. No matter which way you go, the PCIe slave port in RC need to spend the same amount of time to send out 4MB data. And in method #1, EP1 will receive data sooner than EP2/3/4. In method #2, EP1/2/3/4 may receive the data simultaneously (in theory, depending the arbitrary of EDMA TCs on PCIe slave port). But you may not notice the difference depending on your application's requirement.