• State Resolved
  • Locked Locked
  • Replies 6 replies
  • Answers 1 answer
  • Subscribers 97 subscribers
  • Views 1459 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM5728: RAM usage increasing and OOM is envoked

Vijay kumar
Intellectual 540 points
Part Number: AM5728

Tool/software: Linux

Hi,

I am using custom board which is having AM5728 processor.I am running 4  Gstreamer pipelines in the background. The ram of the board is 2GB, and its consumption is increasing as we keep the board for longer time and it is killing the pipelines. I also have 6GB swap partition in it and that is getting filled over time.

the pipeline which i am using is:

gst-launch-1.0 -e v4l2src device="/dev/video3" ! 'video/x-raw,format=(string)NV12,width=720,height=240,framerate=(fraction)30/1' ! videoconvert ! queue ! ducatih264enc bitrate=1000 intra-interval=1 ! queue ! h264parse ! queue ! rtph264pay mtu=800 ! queue ! udpsink host=172.196.129.207 port=36000

regards,

Vijay

  • 0
    TI__Guru* 80190 points

    Hello,

    Why you are using videoconvert this is a software element?
    In your pipeline the format in capsfilter is NV12 , the ducati encoder requires NV12 format as input so there is no need of videoconvert element at all.
    You could remove it.
    If the format that your video source provides is YUY2 / NV12 / YUYV you could use TI VPE element.
    ... v4l2src device=/dev/videoX io-mode=4 ! 'capsfilter' ! vpe num-input-buffers=8 ! ....

    If the VPE is not required also (NV12 use case) the v4l2src io-mode=5 could be set.

    Please take a look in this guide:
    software-dl.ti.com/.../Foundational_Components_Multimedia.html

    It will be nice to try these changes.
    If you are observing the problem again please share the pipelines that are running on background.
    You could add --gst-debug=2 to the pipeline for more debug prints.
    You could try to increase cma to 128M  in the bootargs as well.

    One more note if you are using older version of PSDK like 3.x or lower there was a memleak in ducati enc element. If you are using PSDK 3.x let me know to provide you the patch or move to PSDK 4.x / 5.x.

    Hope this helps.

    BR
    Margarita

  • 0 in reply to Margarita Gashova
    Intellectual 540 points
    Hi,
    Thank you for your reply.

    I modified the pipelines as follows:

    gst-launch-1.0 -e v4l2src device="/dev/video3" io-mode=5 ! 'video/x-raw,format=(string)NV12,width=720,height=288,framerate=(fraction)25/1' ! ducatih264enc bitrate=30000 intra-interval=1 ! queue ! h264parse ! queue ! rtph264pay mtu=800 ! queue ! udpsink host=191.168.1.3 port=36000 &

    what i observed was cpu utilization reduced from 80% to 20% when i run 4 pipelines and functionality was not affected. But still the memory usage is increasing when they are run for long time.

    we are using PSDK 4.01.

    how to increase cma memory ?

    Regards,
    vijay
  • 0 in reply to Vijay kumar
    Intellectual 540 points
    hi,

    I used valgrind to check memory profile in the gstreamer pipeline:

    the output is as follows:

    --17183-- memcheck GC: 23986 nodes, 13111 survivors (54.7%)
    --17183-- memcheck GC: 33921 new table size (stepup)
    ^Chandling interrupt.
    Interrupt: Stopping pipeline ...
    EOS on shutdown enabled -- Forcing EOS on the pipeline
    Waiting for EOS...
    Got EOS from element "pipeline0".
    EOS received - stopping pipeline...
    Execution ended after 0:05:41.516902238
    Setting pipeline to PAUSED ...
    Setting pipeline to READY ...
    Setting pipeline to NULL ...
    Freeing pipeline ...
    --17183-- Discarding syms at 0x6c2993c-0x6c2d840 in /lib/libnss_files-2.23.so due to munmap()
    ==17183==
    ==17183== HEAP SUMMARY:
    ==17183== in use at exit: 4,197,871 bytes in 41,599 blocks
    ==17183== total heap usage: 1,084,089 allocs, 1,042,490 frees, 763,156,382 bytes allocated
    ==17183==
    ==17183== Searching for pointers to 41,400 not-freed blocks
    ==17183== Checked 9,142,360 bytes
    ==17183==
    ==17183== 28 bytes in 1 blocks are possibly lost in loss record 2 of 17
    ==17183== at 0x483D850: malloc (in /usr/lib/valgrind/vgpreload_memcheck-arm-linux.so)
    ==17183==
    ==17183== 560 bytes in 4 blocks are possibly lost in loss record 3 of 17
    ==17183== at 0x4840210: realloc (in /usr/lib/valgrind/vgpreload_memcheck-arm-linux.so)
    ==17183==
    ==17183== 1,772 bytes in 44 blocks are possibly lost in loss record 7 of 17
    ==17183== at 0x483FFA8: calloc (in /usr/lib/valgrind/vgpreload_memcheck-arm-linux.so)
    ==17183==
    ==17183== 4,968 bytes in 32 blocks are possibly lost in loss record 8 of 17
    ==17183== at 0x483D758: malloc (in /usr/lib/valgrind/vgpreload_memcheck-arm-linux.so)
    ==17183==
    ==17183== 3,728,336 (207,116 direct, 3,521,220 indirect) bytes in 12,419 blocks are definitely lost in loss record 17 of 17
    ==17183== at 0x483D850: malloc (in /usr/lib/valgrind/vgpreload_memcheck-arm-linux.so)
    ==17183==
    ==17183== LEAK SUMMARY:
    ==17183== definitely lost: 207,116 bytes in 12,419 blocks
    ==17183== indirectly lost: 3,521,220 bytes in 24,835 blocks
    ==17183== possibly lost: 7,328 bytes in 81 blocks
    ==17183== still reachable: 445,503 bytes in 4,065 blocks
    ==17183== of which reachable via heuristic:
    ==17183== newarray : 1,396 bytes in 46 blocks
    ==17183== suppressed: 0 bytes in 0 blocks
    ==17183== Reachable blocks (those to which a pointer was found) are not shown.
    ==17183== To see them, rerun with: --leak-check=full --show-leak-kinds=all
    ==17183==
    ==17183== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 34 from 12)
    --17183--
    --17183-- used_suppression: 30 dl-hack3-cond-1 /usr/lib/valgrind/default.supp:1218
    --17183-- used_suppression: 4 glibc-2.23-on-SUSE-10.3-(x86) /usr/lib/valgrind/default.supp:1328
    ==17183==
    ==17183== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 34 from 12)


    it says gstreamer is allocating some memory and it is not freeing it .

    is it possible to free memory in pipeline itself?

    Regards,
    Vijay
  • 0 in reply to Vijay kumar
    TI__Guru* 80190 points
    Hello,

    Please make sure that this patch is applied to the gstducatividenc.c file.
    git.ti.com/.../7205d7cb75944e816622d5daca88804819ff16d3

    You could build ducati plugin by following this guide:
    processors.wiki.ti.com/index.php
    but you must use this command MACHINE=am57xx-evm bitbake gstreamer1.0-plugins-ducati instead of MACHINE=am57xx-evm bitbake arago-core-tisdk-image (there is no need to build the full PSDK).
    You could find the source code under //tisdk/build/arago-tmp-external-linaro-toolchain/work/am57xx_evm-linux-gnueabi/gstreamer1.0-plugins-ducati/git-r2.24/git/src folder.
    If the change in the patch is not applied please apply it.
    You could rebuild the ducati library by following "Forced Re-compilation" but replace the opencv with gstreamer1.0-plugins-ducati.
    You could find the new library in /tisdk/build/arago-tmp-external-linaro-toolchain/work/am57xx_evm-linux-gnueabi/gstreamer1.0-plugins-ducati/git-r2.24/git/src/.libs

    .libs is a hidden folder so press crtl+h
    The library name is libgstducati.so so please copy and replace this library on the board and restart.

    Hope this helps.

    BR
    Margarita
  • 0 in reply to Margarita Gashova
    Intellectual 540 points
    Hi,

    Thank you for the reply.
    I followed the steps to compile the ducati library and replaced the .so file.
    Now memory in the RAM is not increasing and it is constant.

    Regards,
    Vijay
  • 0 in reply to Vijay kumar
    TI__Guru* 80190 points
    Hello,

    I am glad that this issue is solved.
    Please if you have new questions/issue open a new e2e thread .

    BR
    Margarita