This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Cache coherency issue

Hello everyone,

I stumbled upon a strange problem yesterday.

Here's the thing :

On C6678, we have a master-slave application (single image on all cores) with SYS/BIOS where the Master notifies (with IPC/notify) the slaves to start working on the data that is in MSMC memory. Each slave work on a non-overlaping part of the data (directly in MSMC) and then notifies the Master that the work is done.

Cache L1P and L1D are enabled, L2 is set as all SRAM (no cache).

When everyone is finished and have notified back the master, the Master read ONE sample of data like a = array[120];

We use the expression view at this point to check the values too.

a) Without the read and with cache enabled : the whole output array has the good values. 100% right.

b) With the read and with cache enabled : the whole array has good values EXCEPT 4 samples starting at the read : array[120] to array[123] values are 0. In total 128-bit is wrong.

c) with the read and with cache L1D disabled for the Master only (enabled for the slaves) : everything is fine again.

I don't get how a single READ affects 128-bit of data in MSMCRAM.

Any ideas ? Am I missing cache coherency operations ?! Is it something else ?

Thank you

Clément

  • Am I missing cache coherency operations ? - Yes, you must be.

    I take it from the line I don't get how a single READ affects 128-bit of data in MSMCRAM.  That array[] is in MSMCM.

    Cache Coherence to external memory space (external from the point of the CorePac is everything that is not L1/L2 of the CorePac.)  Must be maintained by SW.

    If CorePacB caches the space where array[] is located by reading it before CorePacA writes to array[], then CorePacB already has what it sees as valid data in from array[] in L1D.  You would need to Invalidate (and probably writeback if you had modified data that may be outside of array[] but still has the same cache lines in common) the space for array[] before reading it again by CorePacB, otherwise it will grab the cached data.  When CorePacA writes the data to the MSMC it will not update the other cores of it's modification and thus they will still view what they have cached as valid.

    Best Regards,
    Chad 

  • Thank you for the answer, Chad.

    First I'll give you more information to make sure you understand my issue clearly. I'm afraid I wasn't clear enough in the first post.

    Let's say we have an array of 8000 floats in MSMC and 8 cores. For every core L2 cache is disabled, L1D cache enabled.

    CoreA will work on 0 to 1000, CoreB 1000 to 2000, ... CoreH 7000 to 8000

    The execution is as follow

    0) Core A initialise the whole array to zero
    1) CoreA asks other cores to start working (notifysendevent)
    2) cores A to H are working
    3) work is done and CoreA is notified by other cores
    4) I watch the array values directly in MSMC (expression view in CCS)
    5) CoreA does float tmp = array[1230];
    6) I watch the array values directly in MSMC (expression view in CCS)

    at step 4) everything is fine the 8000 values are correct
    at step 6) 7996 values are correct, array[1230] = 0 array[1231] array[1232] array[1233] = 0

    If I disable L1D cache for the Master, step 6) gives 8000 correct values.

    That's why I don't get how the read MODIFIES 128 bit data in MSMCRAM.
    To be clear I get that I'll have to maintain coherency in the code but I need to understand the issue here first.

    Regards,
    Clément

  • What's the data type of array?  It sounds like a cache line from CoreA got evicted (i.e. it wrote out the 0's that was cached into the L1D back out to the MSMC RAM.)  I'd suggest you perform a Cache Writeback Invalidate of Core A's L1D after step 0.)

    Also, in the memory browser make sure you check L1D box and see if this what's resident in cache, when viewing from Core A.

    Best Regards,
    Chad 

  • Ah ok eviction makes sense. I didn't know what it meant exactly.

    Wouldn't it be two lines of L1D ? From the Cache User Guide L1D line size is 64-bit. Here 128 bits of data is set to 0.

    The array is composed of complex data ( 1 32-bit float for Re ,1 32-bit float for Im) and 2 complex values are evicted hence 128 bits.

    I was watching with the expression view (I refresh everytime) in CCS.
    I'll try with the memory browser.

    I'll try your WritebackInvalidate suggestion.

    Thank you for your time,
    Clément

  • Yeah, it could have been 2 lines that got evicted (back to back), the reason I asked for the array type was to make sure it was a Cache line or integer number of Cache lines that appear to have been evicted.

    Best Regards,

    Chad

  • The WritebackInvalidate on the array worked fine.

    Problem solved, thanks for the support.

    Regards,

    Clément