Problem with EDMA and Cache

Are you configuring the MMU before using cache API's ?

Regards

Baskaran

Hello

Yes I configure MMU and Cache like it is shown at the example uartEdma_Cache

regards Tobias

Tobias,

           Can you give more info on the behavior when you say it is not working.

1. is it getting exception in the same line always, if so which one.
2. or is the behavior random - if so give observation on few cases
3. also what are the ranges of memory address that you are trying to access in your application.

Regards

baskaran

Hello

it get allways exception at line 2. But, I think, the exception is at line line 1 because there it is a check to 0, and the receive Buffer is 0 at init

for 3.

I don't understand your question.

I have defined to buffers, like the example shows

#pragma DATA_ALIGN(SrcBuff, SOC_CACHELINE_SIZE_MAX);
#pragma DATA_ALIGN(DstBuff, SOC_CACHELINE_SIZE_MAX);
volatile char SrcBuff[EDMAAPP_MAX_BUFFER_SIZE];
volatile char DstBuff[EDMAAPP_MAX_BUFFER_SIZE];

and have config the MMU like the example uartEdme_Cache shows.

#pragma DATA_ALIGN(PageTable, MMU_PAGETABLE_ALIGN_SIZE);
static volatile unsigned int PageTable[MMU_PAGETABLE_NUM_ENTRY];

/* Definitions related to MMU Configuration. */
#define START_ADDR_DDR                  (0x80000000u)
#define START_ADDR_DEV                  (0x44000000u)
#define START_ADDR_OCMC                 (0x40300000u)
#define NUM_SECTIONS_DDR                (512u)
#define NUM_SECTIONS_DEV                (960u)
#define NUM_SECTIONS_OCMC               (1u)

regards Tobias

I guess that, if the Cache is enabled, the init of the source buffer is not written to the memory, and so the EDMA copy only Zeros to the receive buffer.

If Cache is disabled all thing work.

I have add

 CacheDataCleanBuff((unsigned int)SrcBuff, EDMAAPP_MAX_BUFFER_SIZE);
 

and then I have try to Clean all Cache with

CacheDataCleanAll();

after the init of the source buffer, but both don't change anything

regards Tobias

"it get allways exception at line 2" - which function are you referring here?

Hello

the console  repords

EDMA3Test: Data write-read matching FAILED.

The mismatch happened at index : 2

regards Tobias

Hello

I have done some further tests.

At my Debugger I could see that the datas are written to the CPU memory then copied by

CacheDataCleanBuff((unsigned int)SrcBuff, EDMAAPP_MAX_BUFFER_SIZE);

to the physical memory

then I see that the datas are copied by EDMA to the physical memory of DesBuff

but if I do

CacheDataInvalidateBuff((unsigned int)DstBuff,  EDMAAPP_MAX_BUFFER_SIZE);

No datas are copied to the CPU memory.

What is wrong?

regards Tobias

Hello

now it works, but I don't know why?

#pragma DATA_ALIGN(DstBuff, SOC_CACHELINE_SIZE_MAX);
#pragma DATA_ALIGN(SrcBuff, SOC_CACHELINE_SIZE_MAX);
volatile char DstBuff[EDMAAPP_MAX_BUFFER_SIZE];
volatile char SrcBuff[EDMAAPP_MAX_BUFFER_SIZE];
...
    /* Initalize source and destination buffers */
    for (count = 0u; count < (aCount * bCount * cCount); count++)
    {
        SrcBuff[count] = data++;
        DstBuff[count] = 0xFF;
    }

    // Clean cache to achieve coherence between cached memory and main memory
    CacheDataCleanBuff((unsigned int)SrcBuff, EDMAAPP_MAX_BUFFER_SIZE);
    CacheDataCleanBuff((unsigned int)DstBuff, EDMAAPP_MAX_BUFFER_SIZE);
....
do EDMA job
....
    CacheDataInvalidateBuff((unsigned int)DstBuff,  EDMAAPP_MAX_BUFFER_SIZE);
    /* Match the Source and Destination Buffers. */
    if(TRUE == retVal)
    {
        for(index = 0u; index < (aCount * bCount * cCount); index++)
        {
            if(SrcBuff[index] != DstBuff[index])
            {

Why does it now work?

Is there something wrong with my data alignment?

The SrcBuff is always at the end of the cache, but the DstBuff is only 3 int long and between other variables.

Where is my mistake?

regards Tobias 

CacheDataInvalidateBuff will invalidate the cache contents - it will not be written back to memory.

Whenever you are dealing with cache operations it is better to align the buffer to cache line size (64 bytes in Cortex A8).

Otherwise when cleaning/invalidating additional variables (which fall in same cache line) might get affected.

Hello

I know that, is the alignment not be done with? 

#define SOC_CACHELINE_SIZE_MAX               (64)

#pragma DATA_ALIGN(DstBuff, SOC_CACHELINE_SIZE_MAX);
#pragma DATA_ALIGN(SrcBuff, SOC_CACHELINE_SIZE_MAX);
volatile char DstBuff[EDMAAPP_MAX_BUFFER_SIZE];
volatile char SrcBuff[EDMAAPP_MAX_BUFFER_SIZE];

Now I have the next thing I don't understand.

If I define my buffers like:

#pragma DATA_ALIGN(DestinationBuff, SOC_CACHELINE_SIZE_MAX);
volatile char DestinationBuff[SOC_CACHELINE_SIZE_MAX];
#pragma DATA_ALIGN(SourceBuff, SOC_CACHELINE_SIZE_MAX);
volatile char SourceBuff[SOC_CACHELINE_SIZE_MAX];

then I don't have to read back the Destionation Buffer

void EDMAAppEDMA3Test()
{
    volatile unsigned int index = 0u;
    volatile unsigned int count = 0u;
    EDMA3CCPaRAMEntry paramSet;
    unsigned char data = 1u;
    volatile unsigned int retVal = 0u;
    unsigned int isTestPassed = false;
    unsigned int numEnabled = 0u;
    unsigned int aCount = EDMAAPP_MAX_ACOUNT;
    unsigned int bCount = EDMAAPP_MAX_BCOUNT;
    unsigned int cCount = EDMAAPP_MAX_CCOUNT;

    /* Initalize source and destination buffers */
    for (count = 0u; count < (aCount * bCount * cCount); count++)
    {
        SourceBuff[count] = data++;
    }

    // Clean cache to achieve coherence between cached memory and main memory
    CacheDataCleanBuff((unsigned int)SourceBuff, EDMAAPP_MAX_BUFFER_SIZE);

    /* Request DMA channel and TCC */
    retVal = EDMA3RequestChannel(EDMAAPP_EDMACC_BASE_ADDRESS,
                                 EDMAAPP_DMA_CH_TYPE, EDMAAPP_DMA_CH_NUM,
                                 EDMAAPP_DMA_TCC_NUM, EDMAAPP_DMA_EVTQ);

    /* Registering Callback Function */
    EDMAAppCallbackFxn[EDMAAPP_DMA_TCC_NUM] = &EDMAAppCallback;

    if(TRUE == retVal)
    {
        /* Fill the PaRAM Set with transfer specific information */
.... 
        }

    if(TRUE == retVal)
    {
        /* Need to activate next param */
        if(EDMA3_SYNC_A == EDMAAPP_DMA_SYNC_TYPE)
        {
            numEnabled = bCount * cCount;
        }
        else
        {
            /* AB Sync Transfer Mode */
            numEnabled = cCount;
        }

        for(index = 0u; index < numEnabled; index++)
        {
            IrqRaised = EDMAAPP_IRQ_STATUS_XFER_INPROG;

            /*
            ** Now enable the transfer as many times as calculated above.
            */
            retVal = EDMA3EnableTransfer(EDMAAPP_EDMACC_BASE_ADDRESS,
                                         EDMAAPP_DMA_CH_NUM,
                                         EDMAAPP_DMA_TRIG_MODE);

            /* Wait for the Completion ISR. */
            while(EDMAAPP_IRQ_STATUS_XFER_INPROG == IrqRaised)
            {
            }

            /* Check the status of the completed transfer */
            if(IrqRaised < (int)EDMAAPP_IRQ_STATUS_XFER_INPROG)
            {
...
                break;
            }
        }
    }

// there is now readback from cache nessasary -> why?
  //  CacheDataInvalidateBuff((unsigned int)DestinationBuff,  EDMAAPP_MAX_BUFFER_SIZE);

    /* Match the Source and Destination Buffers. */
    if(TRUE == retVal)
    {
        for(index = 0u; index < (aCount * bCount * cCount); index++)
        {
            if(SourceBuff[index] != DestinationBuff[index])
            {
                isTestPassed = false;
                ConsoleUtilsPrintf("EDMA3Test: Data write-read matching FAILED.\r\n");
                ConsoleUtilsPrintf("The mismatch happened at index : %d\r\n",
                                   ((int)index + 1u));
                break;
            }
        }

        if(index == (aCount * bCount * cCount))
        {
            isTestPassed = true;
            ConsoleUtilsPrintf("EDMA3Test: Data write-read matching PASSED.\r\n");
        }

        /* Free the previously allocated channel. */
        retVal = EDMA3FreeChannel(EDMAAPP_EDMACC_BASE_ADDRESS,
                                  EDMAAPP_DMA_CH_TYPE, EDMAAPP_DMA_CH_NUM,
                                  EDMAAPP_DMA_TRIG_MODE, EDMAAPP_DMA_TCC_NUM,
                                  EDMAAPP_DMA_EVTQ);

        /* Unregister Callback Function */
        EDMAAppCallbackFxn[EDMAAPP_DMA_TCC_NUM] = NULL;

        if(TRUE != retVal)
        {
            ConsoleUtilsPrintf("EDMA3Test: EDMA3_DRV_freeChannel() FAILED.\r\n");
        }
    }

  ...
}

now the Code work like expected.

Where is my misunderstanding?

Thanks for your help.

regards Tobias

The buffers need to be aligned for 64 bytes.

Hello

please see my post above

I have define the buffers

#define SOC_CACHELINE_SIZE_MAX               (64)
#define EDMAAPP_MAX_BUFFER_SIZE              (10u) 

#pragma DATA_ALIGN(DstBuff, SOC_CACHELINE_SIZE_MAX);
#pragma DATA_ALIGN(SrcBuff, SOC_CACHELINE_SIZE_MAX);
volatile char DstBuff[EDMAAPP_MAX_BUFFER_SIZE];
volatile char SrcBuff[EDMAAPP_MAX_BUFFER_SIZE];

for a size of 10 and align them to 64.  But it don't work correct!

regards Tobias

Hello

How could I implement the alignment of a variable?

I use Code Composer Studio and the Ti Compiler.

regards Tobias

Tobias,

       You can refer example files in StarterWare package. For your quick reference i am pasting the sample code below (for 16kb alignment).

#pragma DATA_ALIGN(pageTable, 16384);
static volatile unsigned int pageTable[4*1024];

Regards

Baskaran

Hello,

thanks for your patience.

I do it like you say:

#define SOC_CACHELINE_SIZE_MAX               (64)
#define EDMAAPP_MAX_BUFFER_SIZE              (10u) 

#pragma DATA_ALIGN(DstBuff, SOC_CACHELINE_SIZE_MAX);
#pragma DATA_ALIGN(SrcBuff, SOC_CACHELINE_SIZE_MAX);
volatile char DstBuff[EDMAAPP_MAX_BUFFER_SIZE];
volatile char SrcBuff[EDMAAPP_MAX_BUFFER_SIZE];

But if I look to the Memory with the XDS100 the DstBuff is always to small, only 16 Byte. And the SrcBuff is at the end of the cache, so I could not see how long it is. The filled DstBuff is not copied back to the cache after the DMA operation.

If I define the buffers like

#define SOC_CACHELINE_SIZE_MAX               (64)

volatile char DstBuff[SOC_CACHELINE_SIZE_MAX];
volatile char SrcBuff[SOC_CACHELINE_SIZE_MAX];

and I look to the memory, both buffers are OK and everything work.

What is the problem with my first declaration?

regards Tobias

In the first case the buffers are aligned to 64bytes. So that will fall in different cache line.

In the second case the buffers are not aligned so there is high possibility that both of them fill in same cache line. So as a side effect you may observe that data is getting updated.

In first case you probably may need to clean the buffer to ensure the data goes to DDR.

Hello

sorry, but case 1 don't work !!!!! and the second work because in the second case the buffers are one cache line big!

Yes at case 1 they fall into two cache lines, but the lines are shared with other variables, like I see at XDS100.

And please see my past posts, I do cleaning and invalidating off Cache but I don't work. I don't get my Datas back to cache at case one.

regards Tobias

Tobias,

You may want to read basics of how the ARM cache works. You should be able to find this information in ARM reference manuals. Cache clean writes back the contents of 'dirty' lines in the cache back to the physical memory. Cache invalidate simply marks clear any cache lines in the cache for that memory region. It does not copy back the data from physical memory into cache.

If you are using DMA to transfer data to physical memory, and the data happens to be in the cache as well, you will need to call invalidate to clear it out from cache, so that the subsequent reads will read the data from the physical memory (and not the cache). So the invalidate call is not optional. If you are able to manage without it, you are simply lucky because, as Baskaran mentioned, the data might be getting pulled into cache after EDMA update simply because it spans the same cache line as some other variables.

You need to follow the required cache usage guidelines if you want to have correct operation. The basic things to ensure are:

1. Cache clean/writeback call after writing - if you need to write the data in cache back into physical memory (cache clean & invalidate can be used for additional safety)

2. Cache invalidate before reading any data from physical memory, which may have got modified through non-CPU means (e.g. DMA) after your last read/write.

3. Align to cache line boundary all cacheable memory buffers that you are going to do explicit cache coherence calls on.

4. Align to cache line size, the size of all cacheable memory buffers that you are going to do explicit cache coherence calls on.

If you ensure these rules in general, you would not have any issues. I would still recommend again that you read through the arm cache concepts (or generic cache concepts) before attempting to do cache coherence operations on your own.

Regards,
Mugdha

Hello

thanks again for your patience, I'm here to learn something.

I have read all what you mention, and I have understand the problems of cache usage.

I have open and tested the Starterware example edmaTest. It work->OK

Then I have opened uartEdema_Cache and mixed this with edmaTest and it don't work.

Like you could see at my past post I do the Cache clean and writeback.

I suppose that the alignment of the buffers don't work correct in my case and I don't know why.

As is understand with

#pragma DATA_ALIGN(DstBuff, 64);
volatile char DstBuff[10];

I get a Buffer with 10 Bytes, but at the memory there are 64 Bytes reserved

Are I'm wrong with this?

If I look to the Memory I get the attached picture

 For my understanding that is not correct because the DstBuff is not 64 Byte big.

I could see that the DMA fills the DstBuffer but if I want to read it back to the Cache it don't work because other variables are also changed.

regards Tobias

Hello

I use the API functions of Starterware 2.0.1.1. and there I have

/**
 * \brief   This API clean a section of D-Cache, upto PoC. This API
 *          can be used to make a buffer in D-Cache to be coherent
 *          with the memory. For example, If DMA engine has to access
 *          a memory area for transmitting, to make sure that the 
 *          D-Cache values for the corresponding buffer is written to 
 *          memory, this API can be used.
 *
 * \param   startAddr    Starting address of the buffer to be cleaned
 * \param   numBytes     The number of bytes to be cleaned.
 *
 * \return  None.
 *
 **/
void CacheDataCleanBuff(unsigned int startAddr, unsigned int numBytes)

/**
 * \brief   This API invalidates a section of D-Cache till PoC. With this
 *          API, we can make sure that the next read of the buffer happens
 *          from memory. This is required if any DMA engine has updated
 *          the memory area with any data, other than from the D-Cache.
 *
 * \param   startAddr    Starting address of the buffer to be invalidated
 * \param   numBytes     The number of bytes to be invalidated
 *
 * \return  None.
 *
 **/
void CacheDataInvalidateBuff(unsigned int startAddr, unsigned int numBytes)

The brief of the functions descript exactly what I want to do, but the CacheDataInvalidateBuff don't work at my example.

So either this function does not work properly or the alignment of my buffer is wrong.

I guess the function work correct, so what is wrong with my buffer?

regards Tobias

From the snapshot i see that the DstBuff is allocated with only 12 bytes. Why do you say it is allocated with 64 bytes ?

Hello

thanks, yes that is the problem that I don't understand.

I wrote at my code (copied from the example)

#define SOC_CACHELINE_SIZE_MAX 64
#define EDMAAPP_MAX_BUFFER_SIZE 10

#pragma DATA_ALIGN(DstBuff, SOC_CACHELINE_SIZE_MAX);
volatile char DstBuff[EDMAAPP_MAX_BUFFER_SIZE];

and get this picture / memory.

For my understanding something is wrong? And the Cache could not work correct.

This is the case why I ask "How to align datas".

regards Tobias

Tobias,

I think I understand your question now ...

The DATA_ALIGN only aligns the start address of the buffer to the required alignment. So you can see that this buffer is at the correctly aligned address. But this does not allocate the full 64 bytes to this buffer. Other variables that don't have specific alignment restrictions can always be placed by the compiler at these locations.

You will notice that SrcBuff is at the next aligned start address as per your specification. So only the start address of the buffer will be aligned.

This will not resolve your caching issues. As I mentioned earlier, additional to aligning the buffer start address to cache line boundary, the size of the buffer also has to be separately aligned to cache line size (same as or multiple of cache line size).

Regards,
Mugdha

Hello

OK, that means that I have to define my char buffers like:

#define SOC_CACHELINE_SIZE_MAX 64
#pragma DATA_ALIGN(DstBuff, SOC_CACHELINE_SIZE_MAX);
volatile char DstBuff[SOC_CACHELINE_SIZE_MAX];
#pragma DATA_ALIGN(SrcBuff, SOC_CACHELINE_SIZE_MAX);
volatile char SrcBuff[SOC_CACHELINE_SIZE_MAX]; 

to guarantee that both buffers are at separated and alone at one cache line?

Or there are other ways to lay two buffer with 10 chars too two separated, exclusive Cache areas.

Or do I not need exclusive Cache areas?

 regards Tobias

Hello

thanks to all for your patience.

Tobias