• State Resolved
  • Locked Locked
  • Replies 7 replies
  • Subscribers 100 subscribers
  • Views 1075 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

CMEM memory fragmentation with heap allocation - is it an issue, if all memory gets freed ?

Mike Whittaker
Intellectual 410 points
Other Parts Discussed in Thread: SYSBIOS

The documentation advocates pool allocation for production systems to avoid fragmentation.

However, if using Heap allocation (ie. no pools) if all the memory allocations are performed by just one process, which gets restarted (and hence all buffers automatically freed) - is fragmentation a problem, since CMEM gets all its memory back each time ?

It would be more flexible to omit pool definitions and just let the application request what it needed each time.

But if Heap-based usage always gave rise to fragmentation despite all the CMEM meory being freed from time to time, then it would be necessary to continue to use Pools.

  • 0
    TI__Mastermind 18260 points

    If all CMEM heap memory allocations are freed (either directly, or indirectly as a result of process exit) then there will be no fragmentation.  The CMEM heap allocator coalesces (joins) free blocks as you free them.  For a fresh system, CMEM's heap starts with one big free block, and after all allocated blocks are freed back to the system, there will once again be one big free block.

    CMEM's heap is essentially a port of SysBios's HeapMem module, but without all the configurability or general usage, so documentation about the dynamics of the HeapMem module also apply to CMEM's implementation.

    Regards,

    - Rob

     

  • 0 in reply to Robert Tivy
    Intellectual 410 points

    Well, having dropped the pools definitions from the previously working insmod, we now see:

    CMEM Error: getPool: failed to get a pool fitting a size xxxx

    which suggests that making all CMEM allocations from the heap is not working as expected ...

    [Edit]

    OK, I now need to use useHeapIfPoolUnavailable=[0|1] !!

    Still getting CMEM Error: allocPool: ioctl CMEM_IOCALLOC failed from pool 0: -1

    from my BufTab_create()

    and CMEMK Error: ALLOC: invalid pool (0) passed.

  • 0 in reply to Mike Whittaker
    TI__Mastermind 18260 points

    Mike Whittaker said:

    OK, I now need to use useHeapIfPoolUnavailable=[0|1] !!

    That's right.

    Mike Whittaker said:

    Still getting CMEM Error: allocPool: ioctl CMEM_IOCALLOC failed from pool 0: -1

    from my BufTab_create()

    and CMEMK Error: ALLOC: invalid pool (0) passed.

    The only way I can see the above situation happening is if the code (BufTab_create()) is explicitly calling CMEM_allocPool() (or CMEM_allocPool2()).  The "heap fallback" feature is available only when CMEM_alloc() (or CMEM_alloc2()) is called with params->type == CMEM_POOL.

    I'm not sure if you can confirm this or not, you'd need to know the internals of BufTab_create(), but if you do have access to the source code, check if it is calling CMEM_allocPool().  Or, you could enable user-level debug and see for yourself, by putting this in your .cfg file:
        var CMEM = xdc.useModule('ti.sdo.linuxutils.cmem.CMEM');
        CMEM.debug = true;

    Also, enabling debug for the kernel module will allow useful info to be printed on the console.  To do so, rebuild cmemk.ko for "debug":
        % cd <linuxutils>/packages/ti/sdo/linuxutils/cmem/src/module
        % make debug

    Regards,

    - Rob

     

  • 0 in reply to Robert Tivy
    Intellectual 410 points

    Hmm, that's a bit of a pain - stop using the Pool feature explicitly only to find that the standard API forces its use in any case -  seems to defeat the object a bit :-(

    Could do with a NoReallyUseHeapIfPoolUnavailableIDoMeanThatSoDoItThanks=1

    Looks like the real flag doesn't actually do what it says on the tin.

  • 0 in reply to Mike Whittaker
    Intellectual 435 points

    Hi Mike,

    I'm not sure what platform you are using, or what version of the DVSDK, but on DM36x using DVSDK 4.02.00.06 (linuxutils 2.26.01.02) we are using the CMEM heap without issue.  We use it to allocate video buffers with BufTab_create, as well as other buffers we use for DMA transfers.  I can attest that the heap flag really does work in this environment.  Have you set the memory params on the buffer attributes passed to BufTab_create to the defaults or have you set them explicitly?

    Chris

  • 0 in reply to Mike Whittaker
    TI__Mastermind 18260 points

    Mike Whittaker said:

    Hmm, that's a bit of a pain - stop using the Pool feature explicitly only to find that the standard API forces its use in any case -  seems to defeat the object a bit :-(

    Could do with a NoReallyUseHeapIfPoolUnavailableIDoMeanThatSoDoItThanks=1

    Looks like the real flag doesn't actually do what it says on the tin.

    :) That must be part of the Siri API set?

    I believe that the CMEM_allocPool() API has been deprecated, and was deprecated before this "heap fallback" feature was implemented, so it didn't get enhanced with the feature.

    If you, or code that you call, calls CMEM_alloc() (or CMEM_alloc2(), the block-based version) then this heap fallback feature does work.

    Were you able to see CMEM debug output at all?  Since I don't know your code I'm still actually guessing as to the root cause of your issue.  If your issue is not due to explicit calls to CMEM_allocPool() then I would need to investigate elsewhere.

    Regards,

    - Rob

  • 0 in reply to Robert Tivy
    Intellectual 410 points

    Thanks for your responses chaps - will update this when this issue gets another time-slice for investigation, until when the pools will remain ...