This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Memset/Memcpy Suspected Issues

Hi,

 

I am using memcpy and memset to initialize very large data structures.  I mean, they seem large to me :) 

Every loop, I initialize a struct of 32384 bytes to zero.  Then I memcopy chunks of 640 bytes into it and also, set individual variables by assignment. 

What I see, is it seems like sometimes the memset trickles into the next variables.  I can see this is I tell memset to assign something other than zero.  Are there any suspected culprits for this behavior?  I also sometimes see that the data copied is incorrect.  For instance, a variable will have an invalid number, like the bytes copied were maybe shifted?

Here is a better description of my structs.

I have an array[5000] of largeStruct.  sizeof(largeStruct) = 32384

Inside each largeStruct is an array[50] of smallerStruct.  sizeof(smallerStruct) = 640.  The remaining space is misc variables.


I tried to make largeStruct and smallerStruct cache aligned to 128 bytes.  I also placed smallerStruct at the beginning of largeStruct so it will start at a cache aligned position.

 

I read about maybe a 16K roll over problem, but I am under 16K.  Maybe this is similar but I can't put my finger on the solution.

 

I also get this warning when I compile, but I read somewhere else that it is ok to ignore it :

"

<Linking>

"./configPkg/linker.cmd", line 357: warning #10237-D: split placement (>>)   ignored for ".myLocalMemory":  split run placement for this section is not  permitted

'Finished building target: PXMTracking.out'

"

 

 

Thanks,
brandy

 

I am using C6678, CCS Version: 5.1.0.09000

 

  • One thing to consider ... memcpy presumes the two memory ranges do not overlap.  If you need to copy overlapping memory ranges, use memmove instead.  So, try using memmove instead, just to see if that makes it work.

    BrandyJ said:
    seems like sometimes the memset trickles into the next variables

    BrandyJ said:
    I also sometimes see that the data copied is incorrect.

    Sometimes, and not every time?  Intermittent behavior like that usually means there is a problem with the hardware, and not the software.

    Thanks and regards,

    -George

  • Hi George,

     

    By sometimes, I mean that most of the time my algorithms are working.  But after say 262 loops, an error shows up.  When I look at the variables, it looks like either the memset/memcpy overwrite the wrong data.  I agree since it works some of the time, maybe it is still a caching/multicore issue.  But it appeared when I changed my struct from using vectors from stl (so small) to making static allocations for the vectors as arrays (large amounts of memory).  I just wanted to make sure there wasn't a limitation of sorts that I was hitting regarding the size of the memset and memcpy I was making.  Or that the compilier warning I showed you wasn't some how effecting my program. 

    I am running out of ideas to fix the problem, so I wanted to see what you thought.  At worst case, maybe you suggest something that triggers something new in my head to check or try.

    Thank you, but my memory sections are not overlapping.  I always copy a single struct entirely, not pieces.  I use the sizeof(largeStruct) or sizeof(smallerStruct) to specify the size.

    Thanks,
    brandy

  • I should add, not every loop uses every struct in the array.  Loop 1 might activate structs 0 - 100, Loop 2 might add 101 - 150 as active and deactivate 5-20 from Loop 1 etc.

    So the array is used a circular buffer which is another reason why I was wondering if there was a memory boundry being crossed or a counter being rolled over in memset or memcpy. 

     

  • Hello,

     

    I am still have little progress with this problem.  I tried to align everything I could think of to 128 bytes and that did not work either.

    What exactly does the compiler warning mean?  Could it be causing some odd behavior?

     

    "./configPkg/linker.cmd", line 357: warning #10237-D: split placement (>>)

       ignored for ".myLocalMemory":  split run placement for this section is not

       permitted

     

    Thanks,
    Brandy

  • Split placement is where the linker takes (for instance) the various .text sections from the input files, and rather than combine them into one monolithic output section for placement, spreads it across two or more memories.  Splitting is not allowed for certain sections like .cinit, .stack, and .sysmem, which must be contiguous.  Other sections may be unsplittable depending on the target. What target ISA are you using?

    The other reason a section might be unsplittable is quite a bit more cryptic.  The simplified version is that it is probably in a GROUP that must be placed late, but split objects are not allowed to be placed late.  You will probably need to move .myLocalMemory out of the group it is in, or change >> to > to disable splitting.

  • BrandyJ said:
    By sometimes, I mean that most of the time my algorithms are working.  But after say 262 loops, an error shows up.

    If the problem consistently occurs on iteration 262, then it is probably a straightforward software issue.  But if the problem doesn't occur consistently, then there must be some race condition, or even a hardware issue.

    BrandyJ said:
    I just wanted to make sure there wasn't a limitation of sorts that I was hitting regarding the size of the memset and memcpy I was making.

    I'm not aware of any such limitation.

    Thanks and regards,

    -George

  • Hi,

    Were you able to solve your problem? If yes, how did you do it?

    I have a similar problem. After a memcpy, the content of my src pointer is placed in dest - 4 bytes. As an example, if my destination parameter for memcpy is 0x913B0C6C, after memcpy returns, the data starts at 0x913B0C68.

    In my case, the src-parameter is 8-Byte-aligned, the dest-parameter is only 4-byte-aligned. Both point to the same type of struct within different types of structs.

    Any ideas?

    Kind regards,

    Roman

  • What is the alignment of the structure being copied?  For structures which contain only scalar members, this is the same as the size of the largest scalar.  If the structure contains other structures, then you have to apply this rule recursively.  And the alignment of the parent structure is max(parent structure, child structure).  (There might be some exceptions to this, but I can't think of any.)  

    I suspect the alignment the structure being copied is more than 4, which makes the 4-byte alignment, and the 4-byte aligned destination address (it ends with 0xC) both suspicious.

    Thanks and regards,

    -George

  • Roman,

    In the end, I discovered that it was not the memset but a cache writeback/inv.  Even though things were aligned to 128 bytes, sometimes I was invalidating and writing back inside the struct where it was not aligned and it was over writing in to the beginning of the other structure. 

    Sorry, I should have updated my post.   I hope TI can help with your problem. 

    Brandy

  • George,

    yes, you were right, the struct has three uint64 members, so the compiler assumes it to be aligned to 8 bytes.

    The problem in my case was that I have to copy the struct to a contiguous memory region in order to send it via ethernet. The function where I experienced the problem just takes a pointer to the next "free" position in this send-buffer. At the moment, we cannot guarantee any alignement of this buffer.

    I resolved the problem by assembling the struct locally (on the stack) and copying (casting memcpy-parameters to (uint32*) ) it to the right place.

    Thanks for your help,

    Roman