DMA transfer over PCIe

I have following update from my side on this:

-          I referred document tms3206657.pdf and I found the address of EDMA3CC was 0x02740000(table 2-2, page 23), not 0x02700000. Then I replaced the address and tried but internal and PCIe , both dma transfer didn’t succeed.

-          Then I read the value of each register from cavium dirver file itself. So I read every register immediately after writing to it. I found all the values matching Here.

-          Then I connected DSP with JTAG and read the memory of all those registers. Here is the problem. I didn’t find any value of the register matching with what I wrote

So that means in pciedemo.c file the address was not correct mapped to 0x02740000.  Following is how 0x02740000 is mapped in pciedemo.c, function- HAL_readDMA():

#define EDMA_TPCC0_BASE_ADDRESS      0x02740000  è Originally this was 0x02700000. Not sure why it was 0x02700000. May be pcidemo example was for c6678.

#define DMAQNUM0                     0x0240 

#define ESR                          0x1010

#define EESR                         0x1030                

#define IESR                         0x1060

#define IPR                          0x1068

#define ICR                          0x1070

#define PARAM_0_OPT                  0x4000

#define PARAM_0_SRC                  0x4004

#define PARAM_0_A_B_CNT              0x4008

#define PARAM_0_DST                  0x400C

#define PARAM_0_SRC_DST_BIDX         0x4010

#define PARAM_0_LINK_BCNTRLD         0x4014

#define PARAM_0_SRC_DST_CIDX         0x4018

#define PARAM_0_CCNT                 0x401C

HAL_readDMA()

{

…………

…………

                /* Temporarily re-map IB region 3 from DDR memory to EDMA registers */

                iowrite32(EDMA_TPCC0_BASE_ADDRESS, pReg + IB_OFFSET(3)/4); 

                pReg = (uint32_t*)gss_dev->ddrVirt;   /* Now it points to the start of EDMA_TPCC0_BASE_ADDRESS */

…………

…………

}

Does this line of code really re-map the ddrVirt to EDMA register base? If yes then All my DSP register should have been updated. Can you confirm this from pciedemo.c code? Iw ill update the same in e2e community also.

Thanks

Kashif 

Hi Kashif,

MCSDK PCIe Linux host loader EDMA-Interrupt boot example demo code(pciedemo example) supports C6670 and C6678 platform only. C6657 platform support ddrinit/helloworld/post demos only.

Please take a reference of C6670/78 EDMA-Interrupt boot example demo code and develope your application.

Yes, IB_OFFSET(3) was used for DDR originally, we temporarily changed it to point to EDMA_TPCC0.

Please take a look at below thread:

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/291330/1015977.aspx#1015977

Thanks,

Hi Kashif,

Any Update. Please help us to close this thread.

Thanks,

Hi Padmaja,

Here is update after our meeting. The subsequent transfer issue was resolved and we were able to transfer data over PICIe using eDMA. Now I am facing another issue. Once cavium octon host gets MSI interrupt, it transfers all the buffers from DSP over PCIe EDMA but after first interrupt , no further interrupts are received by host while DSP sends interrupt in every 5 ms. I confirmed DSP interrupts by probing the interrupt pin.

So it seems HAL_readDMA() function disables the msi interrupts(Not sure why?).

In your pciedemo code I can see following when calling HAL_readDMA():

static irqreturn_t ISR_handler(int irq, void *arg)

{

                uint32_t status = HAL_CheckPciInterrupt();

                if (status == 1) {

                                HAL_readDMA(DDR_START, rData, DMA_TRANSFER_SIZE, 1);     /* Move from DSP to GPP */

                                HAL_PciClearDspInterrupt();

                                return IRQ_HANDLED;

                }

                return IRQ_NONE;

}

It seems you didn’t use MSI interrupt in your example code. I have two questions now:

1)      Can I use msi interrupts with eDMA transfer.

2)      If yes, why msi interrupts appears to be disabled after first eDMA transfer

Hi Kashif,

Ans1: Yes.

Ans2: MSI interrupts is not disabled after the first EDMA transfer. In pciedemo code interrupt is triggered once only.

If you want to use the MSI interrupts continuously means, you need to update EOI bits on IRQ_EOI register. (Example: you using MSI0 means write 0x4 to IRQ_EOI register /* end of MSI0, event number=4 */) Refer section 3.1.15 End of Interrupt Register (IRQ_EOI) on PCIe user guide.

Please refer the following thread. Hope it helps.

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/161475.aspx#589428

Thanks,

Ganapati,

I wrote "0x4" to IRQ_EOI but this didn't help at all. Please note my configuration is different from the example you shared (Though very similar). In your  example it is the host who generates periodic interrupt to DSP while in my example DSP generates MSI interrupt in every 5 ms to host and If I use eDMA transfer(dsp-> host direction) then host does not receive any interrupt from DSP after the first one.

If I don't use eDMA but use directly copy the data from DSP (dsp-> host direction) then host continuously receives periodic interrupts. That is very strange what eDAM transfer does that stops MSI interrupts.

I need help in debugging this. First how to confirm if DSP has actually generated second interrupt or not? secondly if generated the second interrupt then why host did not receive it. I would like to get the first part confirmed with the help of TI.

Hi Kashif,

Please take a look at below threads:

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/199151/709929.aspx#709929

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/p/166202/606342.aspx#606342

Thanks,

I will check these links but I am looking for the answer why these MSI interrupts work fine when I don't use eDMA. Why the problem comes only in eDMA.

Also at this stage I don't know the msi problem is on DSP side or the host side. My objective here is to confirm whether DSP is sending interrupt to host or not and I am not getting pointed answers for that.

Ganapati,

i just checked the link and link says "Write 0x1 to each bit of the MSI0_IRQ_STATUS will clear the status of the MSI interrupt. It should be done before you exit the ISR and make the MSI vector available for the next triggering.".

While in your example code pciedemo.c you cleared the interrupt by following function:

void HAL_PciClearDspInterrupt(gauss_device_t *gss_dev)
{
uint32_t *pReg = (uint32_t *)gss_dev->regVirt;
iowrite32(1, pReg+EP_IRQ_CLR/4);
}

Hi Kashif,

In pciedemo.c example code, HAL_PciClearDspInterrupt(gauss_device_t *gss_dev) function cleared the Endpoint Interrupt Request Clear Register (EP_IRQ_CLR).

You need to implement the code for clear the MSI 0 Interrupt Enabled Status Register (MSI0_IRQ_STATUS).

Thanks,

Ganapati,

I did clear MSI0_IRQ_STATUS but this too doen't help. As I told you earlier, the same code works fine if I don't use eDMA. We have to understand the difference when I use eDMA and when I don't use eDMA. Why the same vode works fine if I don't use eDMA.

We need to have call with TI and discuss the problem to make it faster.