This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Cache line alignment of struct members?

HI, everyone.

I'm having a problem on how to align struct member to cache line boundaries.

I'm able to align the beginning of the struct to arbitrary boundaries, but I'm not quite sure how to align struct members.

Is there a way to align the members to large boundaries such as cache lines?


Thank you

Dehuan

  • Hi,

    Thanks for your post.

    In general,  to align any buffer to a given boundary the C6000 compiler supports the #pragma DATA_ALIGN() function. This function can be used to align buffers to the cacheline boundary without allocating extra memory.

    However, the requirements of having buffers aligned to cache line boundary and having the the byte count as multiple of cache line size is more critical when if the application has GPP or the DSP using DMA to access shared buffers. This rule is part of the XDAIS standard and is very well documented in the wiki on Cache management given below:

    http://processors.wiki.ti.com/index.php/Cache_Management#DMA-Related

    In the Training section of TI.com, there is a training video set for  the C6474 which uses three (3) of the C64x+ cores. It may be helpful for you to  review several of the modules. But in particular, the Memory and Cache Module  will apply to handling the cache coherency issues. You can find the complete  video set at

    http://software-dl.ti.com/public/c6474/C64x_Memory_Cache/index.html

    Thanks & regards,

    Sivaraj K

    -------------------------------------------------------------------------------------------------------

    Please click the Verify Answer button on this post if it answers your question

    -------------------------------------------------------------------------------------------------------

  • Thanks Sivaraj, that make sense to me.

    Our usage is that, GPP passes a struct to DSP, DSP uses most of the fields as inputs, and writes the outputs into a few fields in the same struct. Then the GPP reads the output.

    Currently out DSP only writes-back and invalidates a minimal amount of that struct. However, even though the beginning and end of that struct is cache aligned, the fields inside it is not necessarily aligned.

    A related question will be, on a GPP+DSP platform, does XDIAS allow input-to/output-from DSP be the same buffer block? Are there modules that handles this kind of usage?

    Thank you

    Dehuan

  • Hi,

    Thanks for your update.

     All XDIAS standard rules are documented in the wiki which i shared in my previous post. Kindly refer it.

    Thanks & regards,

    Sivaraj K

  • Dehuan Xin94 said:

    Our usage is that, GPP passes a struct to DSP, DSP uses most of the fields as inputs, and writes the outputs into a few fields in the same struct. Then the GPP reads the output.

    Currently out DSP only writes-back and invalidates a minimal amount of that struct. However, even though the beginning and end of that struct is cache aligned, the fields inside it is not necessarily aligned.

    "Fields inside [struct] is not necessarily [cache] aligned" is kind of a trick statement. If fields are types like integer or double, they are aligned. Because even if placed into a structure they are still aligned at their natural sizes. And if they are aligned at natural sizes, they can't cross cache line. The only case when a structure member can cross cache line is when it's a vector of values, array or nested structure. In order to align such fields you have to use padding, e.g.

    struct {

    union { int i; char padding[CACHE_LINE_SIZE]; } i;

    int this_one_is_aligned[56];

    } ...

  • Andy,

    This is exactly our case.

    I searched on the web and saw similar answers that refers to "nameless union"s.

    If I understand it correctly, is this method ensuring aligning fields by adding padding before what to be aligned, assuming that the starting address of the struct is already cache aligned? And then the alignment of the starting address of the whole struct is a separate issue that can be addressed by other techniques such as pragmas?

    Thank you

    Dehuan

  • Except that TI compiler doesn't seem to support nameless union/struct's, so you have to name it. And it's "yes" to both questions.

  • Thank you very much, Andy.

    You have fully answered all my questions.

  • Andy Polyakov said:

    If fields are types like integer or double, they are aligned. Because even if placed into a structure they are still aligned at their natural sizes. And if they are aligned at natural sizes, they can't cross cache line.

    I realized that there is a risk that what I really meant didn't make across.  I kind of interchanged cache-line alignment with not crossing cache line without explanation. Inter-processor communication is all about flushing cache to shared memory, right? But when it comes to flushing a value such as integer or double, it doesn't matter where within the cache line it resides, because whole line has to be flushed anyway. This is under assumption that such value doesn't cross cache line (it doesn't), so that you don't have to flush pair of adjacent cache lines. Which brings us to above statement, but to its true meaning, i.e. for structure fields like integer or double, you don't have to think about their alignment within cache line, because such fields never cross line boundary.

  • If I understand correctly, the point you are trying to make is that, a natural type can never cross a cache boundary, hence it will never happen that cache coherence operations only flush part of it back and leave the rest in the next cache line.

    However, the scenario I'm concerned about is that, when two processors are trying to update adjacent fields of a struct, and when those fields reside in the same cache line, then one processor's cache flush may undo the the other processor's update.

    The solution I'm seeking, to eliminate this risk, is a way to ensure that those adjacent fields never resides in the same cache line.

  • Dehuan,

    FYI, when you mark a post as a Verified Answer, I assume that means you are finished with the thread and no more information is needed. If your original question has been answered but you have a new question, you might mark this one as answered and then start a new thread with your new question. Normally, I do not look at threads marked as Answered, but I have noticed yours continues to be near the top of the list telling me that it is still active. It is kind of you to mark the posts as Answered, but it might confuse some of the support team and some of the other users like yourself.

    You must live within the constraint of cache-line sizes. One thing this means is that no two processes or processors should be accessing and updating portions of the same memory within a cache-line size. The multi-processing environment tools like OpenMP may features to help you control this, but it is otherwise up to you to make sure you do not mix things that can be on the same cache line. Do this using struct alignment and preventing the use of that same  struct by multiple processes at the same time. If that is impractical because of the use of very large structs that need to have different parts shared, then you will have to find a way to avoid the use by different processes of anything that is on the same cache line.

    It is not clear what your constraints are, other than that you have a potential problem. The underlying problem is trying to share things on too fine of a boundary, that is sharing things that may be on the same cache line. You cannot do that unless you make those areas to be uncacheable, which will make a very large region uncacheable.

    I think your original question was how to set the alignment of a struct, and I think that has been answered. You were then asking about how to align members of a struct, and that is probably not possible. If you want to go deeper into how to use the compiler tools to implement some special alignment, please post a new question to the E2E C/C++ Compiler Forum where that can be discussed with the compiler team.

    If you have new questions for this thread, you may want to start a new thread instead of continuing this one, or if it is still related to the original question then you may want to remove the Verified Answer marks for those posts above.

    I hope this has helped in some way.

    Regards,
    RandyP

  • software-dl.ti.com/.../index.html

    Not Found

    The requested URL /public/c6474/C64x_Memory_Cache/index.html was not found on this server.