This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

PCIe transfer rate using EDMA in the filetest demo

Hi, 

We have developped a custom board using the C6655 DSP. We have ported the Desktop Linux SDK to the ARM platform and also added support for the C665x DSP.

We finally managed to get DMA to work and starting comparing the different PCIe transfer rate using the filetest demo. Here is the returned transfer rates from the tool for both the DMA and memcpy test:

Payload size(bytes) DMA read rate (Mbytes / s)  DMA write rate (Mbytes / s)  memcpy read rate (Mbytes / s)  memcpy read rate (Mbytes / s) 
128 1 1 11 29
256 3 3 16 34
1024 12 11 25 37
4096 41 30 28 38
65536 124 60 29 37

I was a bit surprised that the DMA read/write rates were so low when using small payloads.

Is this expected? 

If so, what is the causing the overhead when using DMA with small buffers?

Is there anything that should be changed in the filetest demo that could accelerate the DMA transfer for smaller buffers?

Best regards,

- David

  • Hi,

    Welcome to the TI E2E forum. I hope you will find many good answers here and in the TI.com documents and in the TI Wiki Pages (for processor issues). Be sure to search those for helpful information and to browse for the questions others may have asked on similar topics (e2e.ti.com).

    It is not a expected PCIe throughput test result with EDMA. C6655 have single DSP core only. Desktop Linux support C6678 devices only.

    The Linux host loader Example only works on Linux PC. Please refer “\ti\mcsdk_x_xx_xx_xx\tools\boot_loader\examples\pcie\docs\Readme.pdf” document for more information.

    Thanks,
  • Like I said in my previous post, we have ported the Desktop linux SDK to support the C6655. We have no problems using the SDK to transfer data to and from the DSP using either the memcpy or the CMEM approach. We managed to get EDMA to work simply by setting the EDMA_TPCC0_BASE_ADDRESS base address to 0x0274000. However, I am getting poor transfer rate when using small payloads.

    Also, I have tried to compile the Linux host loader example for the C6655. I have enabled the EDMA_INTC_DEMO and the EVMC6657L defines but I am getting the following compilation errors:

    /extra/opt/ti/mcsdk_2_01_02_06/tools/boot_loader/examples/pcie/linux_host_loader/pciedemo.c: In function 'init_module':
    /extra/opt/ti/mcsdk_2_01_02_06/tools/boot_loader/examples/pcie/linux_host_loader/pciedemo.c:1640:12: error: 'pcieInterrupt' undeclared (first use in this function)
       pushData(pcieInterrupt, 0, &bootEntryAddr);
                ^
    /extra/opt/ti/mcsdk_2_01_02_06/tools/boot_loader/examples/pcie/linux_host_loader/pciedemo.c:1640:12: note: each undeclared identifier is reported only once for each function it appears in
    /extra/opt/ti/mcsdk_2_01_02_06/tools/boot_loader/examples/pcie/linux_host_loader/pciedemo.c:1573:20: warning: ignoring return value of 'pci_enable_device', declared with attribute warn_unused_result [-Wunused-result]
       pci_enable_device(PCIE_DEV);
                        ^
    /extra/opt/ti/mcsdk_2_01_02_06/tools/boot_loader/examples/pcie/linux_host_loader/pciedemo.c:1597:14: warning: ignoring return value of 'request_irq', declared with attribute warn_unused_result [-Wunused-result]
       request_irq(irqNo, ISR_handler, IRQF_SHARED, "TI 667x PCIE", &dummy);
                  ^
    make[2]: *** [/extra/opt/ti/mcsdk_2_01_02_06/tools/boot_loader/examples/pcie/linux_host_loader/pciedemo.o] Error 1
    make[1]: *** [_module_/extra/opt/ti/mcsdk_2_01_02_06/tools/boot_loader/examples/pcie/linux_host_loader] Error 2
    

    So, I can't test using that demo either.

    Regards,

    - David

  • Hi,

    Linux host loader pciedemo(EDMA_INTC_DEMO test) required "pcieInterrupt_6657.h" file. Refer section 7 "EDMA-Interrupt Boot Image" on “\ti\mcsdk_x_xx_xx_xx\tools\boot_loader\examples\pcie\docs\Readme.pdf” document.

    Note: TI not provide pcieboot_localreset and pcieboot_interrupt CCS project for C6657 devices. Refer C667x device example and develop the project for C6657 device. Some customers are tested this example on C6657 devices.

    Thanks,
  • Hi,

    I managed to modify the pcieboot_interrupt CCS project for the C6657 and use the linux_host_loader example to transfer PCIe data using EDMA. Now, I wanted to port the logic from the pcieboot_interrupt example (the interrupt handler) to SYS/BIOS.

    I need to map the PCIEXpress_Legacy_INTA (Secondary Interrupts for C66x CorePacs) to one of the interrupt controller output. From what I can understand, this should be done using the CpIntc API and use the Hwi API to map the returned event Id to an actual ISR function. However, I can't get this to work.

    I am using the following code to initialize and trap the ISR. Any help in investigating my issue will be greatly appreciated.

    Regards,

    #define PCIE_EP_IRQ_SET              0x21800064
    #define PCIEXpress_Legacy_INTA               50
    
    #define HOST_INT  3
    #define HWI_INT  4
    
    void MyIsrFunction(UArg intNum)
    {
      printf    ("DSP receives interrupt from host.\n");
    
      Hwi_clearInterrupt(intNum);
      *((uint32_t *)PCIE_EP_IRQ_SET) = 0x1;
      printf    ("DSP generates interrupt to host.\n");
    
    }
    
    void EnableSysInterrupt()
    {
      UInt32 cookie = Hwi_disable();
    
      CpIntc_mapSysIntToHostInt(0, PCIEXpress_Legacy_INTA, HOST_INT); 
      CpIntc_dispatchPlug(PCIEXpress_Legacy_INTA, &MyIsrFunction, PCIEXpress_Legacy_INTA, TRUE);
      CpIntc_enableHostInt(0, HOST_INT);
      CpIntc_clearSysInt(0, PCIEXpress_Legacy_INTA);
    
      int eventId = CpIntc_getEventId(HOST_INT);                                          // get the event id for host interrupt 8
    
      Hwi_Params params;
      Hwi_Params_init(&params);
      params.arg = HOST_INT;
      params.eventId = eventId;      
      params.enableInt = TRUE;       
      Hwi_Handle hdl = Hwi_create(HWI_INT, &CpIntc_dispatch, &params, NULL);      // plug interrupt vector 4, make sure function here is the CpIntc_dispatch
    
      Hwi_enableInterrupt(HWI_INT);
    
      CpIntc_enableAllHostInts(0);
    
      /* Restore interrupts */
      Hwi_restore(cookie);
    
      printf("sys(%d) -> host(%d) -> eventId(%d) -> Hwi(%d)\n", PCIEXpress_Legacy_INTA, HOST_INT, eventId, HWI_INT);
    }

  • Hi,

    I eventually figured it out.
    My ISR handler needed to clear the PCIe interrupt. Afterwards, everything worked as expected.

    Regards,
    - David