• State TI Thinks Resolved
  • Locked Locked
  • Replies 32 replies
  • Answers 3 answers
  • Subscribers 101 subscribers
  • Views 13797 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Linux/AM5728: AM5728 Linux ducatih264enc Gstreamer plugin memory leak

Jian Zhou
Intellectual 1930 points
Part Number: AM5728

Tool/software: Linux

Hi, Expert

My customer reported that they found the ducatih264enc Gstreamer plugin memory leak base on the PDK 03.03.00.04 on AM5728 EVM, could please have a check?

  • 0
    TI__Guru* 80190 points
    Hello,

    Are they observe this leak with videnc2test or only with gstreamer?
    Are they observe this only when the ducatih264enc is used ?
    Are they executing pipeline or they observe this with application?

    BR
    Margarita
  • 0 in reply to Margarita Gashova
    TI__Intellectual 1930 points
    Thanks for your responds.
    Customer only use the gstreamer pipeline , and they do not found the memory leak without the ducatih264enc plugin.
  • 0 in reply to Margarita Gashova
    Intellectual 300 points

    Hi,Expert

     when i use with application is memory leak.

    v4l2 for cap data, v4l2 -> appsrc -> ducatih264enc(ti for encode ) -> filesink  {this way is memory leak}

     v4l2-> appsrc -> filesink {this way not find memory leak}

    /*
     *
     * compile command  arm-linux-gnueabihf-gcc app-camera.c -o app `pkg-config --cflags --libs gstreamer-1.0 gstreamer-base-1.0 gstreamer-app-1.0`
     *
    */
    #include <linux/videodev2.h>
    #include <sys/ioctl.h>
    #include <sys/mman.h>
    #include <unistd.h>
    #include <stdio.h>
    #include <stdint.h>
    #include <errno.h>
    #include <fcntl.h>
    #include <stdlib.h>
    #include <string.h>
    #include <gst/gst.h>
    #include <gst/app/gstappsrc.h>
    #define waylandsink
    #define v4l2src
    typedef struct _App App;
    struct _App
    {
        GstElement *pipeline;
        GstElement *appsrc;
        GstElement *encode;
        GstElement *parse;
         GstElement *sink;
        GstElement *decode;
        GstBus *bus;
        GstMessage *msg;
        GMainLoop *loop;
        guint bus_watch_id;
    };
    App s_app;
    int ret, idx, fd;
    #define NBUF 3
    #define FRAME_SIZE 152064 //352*288*1.5
    int width = 352, height = 288;
    void *buffer_addr[NBUF];
    int size[NBUF];
    static void
    feed_data (GstElement * appsrc, guint size, App * app)
    {
        printf("feed-data....\n");
        struct v4l2_buffer buf;
        /* Dequeue one buffer */
        memset(&buf, 0, sizeof(buf));
        buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
        buf.memory = V4L2_MEMORY_MMAP;
        if(-1 == xioctl(fd, VIDIOC_DQBUF, &buf)) {
            perror("Queue Buffer");
            return FALSE;
        }
        idx = buf.index;
        GstBuffer *buffer;
        GstFlowReturn ret;
        buffer = gst_buffer_new_allocate(NULL,FRAME_SIZE,NULL);
        GstMapInfo info;
        gst_buffer_map(buffer,&info,GST_MAP_WRITE);
        unsigned char * buff = info.data;
        memcpy(buff,buffer_addr[idx],FRAME_SIZE);
        gst_buffer_unmap(buffer,&info);
        g_signal_emit_by_name(app->appsrc,"push-buffer",buffer,&ret);
        printf("ret:%d\n",ret);
        if (ret != GST_FLOW_OK) {
            /* some error, stop sending data */
            printf("push error...\n");
            return FALSE;
        }
        gst_buffer_unref(buffer);
        memset(&buf, 0, sizeof(buf));
        buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
        buf.memory = V4L2_MEMORY_MMAP;
        buf.index = idx;
        if(-1 == xioctl(fd, VIDIOC_QBUF, &buf)) {
            perror("Queue Buffer");
            return FALSE;
        }
        return TRUE;
    }
    int xioctl(int fd, int request, void *arg)
    {
        int r;
        do r = ioctl (fd, request, arg);
        while (-1 == r && EINTR == errno);
        return r;
    }
    int init_device(int fd) {
        unsigned int i;
        struct v4l2_capability caps;
        struct v4l2_format fmt;
        struct v4l2_requestbuffers req;
        struct v4l2_buffer buf;
        /* Check for capture device */
        memset(&caps, 0, sizeof(caps));
        if (-1 == xioctl(fd, VIDIOC_QUERYCAP, &caps)) {
            perror("Setting Pixel Format");
            return 1;
        }
        printf("Driver: %s\ncaps: %8x", caps.driver, caps.capabilities);
        if (~caps.capabilities & V4L2_CAP_VIDEO_CAPTURE) {
            printf("Not a capture device");
            return 1;
        }
        /* Set capture format to UYVY */
        memset(&fmt, 0, sizeof(fmt));
        fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
        fmt.fmt.pix.width = width;
        fmt.fmt.pix.height = height;
        fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_NV12;
        //  fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_UYVY;
        fmt.fmt.pix.field = V4L2_FIELD_NONE;
        if (-1 == xioctl(fd, VIDIOC_S_FMT, &fmt)) {
            perror("Setting Pixel Format");
            return 1;
        }
        printf("Selected Camera Mode:\n" "  Width: %d\n" "  Height: %d\n" "  Field: %d",
            fmt.fmt.pix.width, fmt.fmt.pix.height, fmt.fmt.pix.field);
        printf("  PixFmt = %c%c%c%c\n",
               fmt.fmt.pix.pixelformat & 0xFF, (fmt.fmt.pix.pixelformat >> 8) & 0xFF,
               (fmt.fmt.pix.pixelformat >> 16) & 0xFF, (fmt.fmt.pix.pixelformat >> 24) &0xFF);
        /* Currently driver supports only mmap buffers
         * Request memory mapped buffers */
        memset(&req, 0, sizeof(req));
        req.count = NBUF;
        req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
        req.memory = V4L2_MEMORY_MMAP;
        if (-1 == xioctl(fd, VIDIOC_REQBUFS, &req)) {
            perror("Requesting Buffer");
            return 1;
        }
        printf("Total buffer num %d\n", req.count);
        for (i = 0; i < req.count; i++) {
            memset(&buf, 0, sizeof(buf));
            buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
            buf.memory = V4L2_MEMORY_MMAP;
            buf.index = i;
            if (-1 == xioctl(fd, VIDIOC_QUERYBUF, &buf)) {
                perror("Querying Buffer");
                return 1;
            }
            /* Memory map all the buffers and save the addresses */
            buffer_addr[i] = mmap(NULL, buf.length, PROT_READ | PROT_WRITE,
                                 MAP_SHARED, fd, buf.m.offset);
            //buffer_addr[i]=(void*)malloc(FRAME_SIZE);
            size[i] = buf.length;
            printf("Address %p, size %d, image size: %d \n", buffer_addr[i], buf.length, buf.bytesused);
            /* Queue the buffer for capture */
            memset(&buf, 0, sizeof(buf));
            buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
            buf.memory = V4L2_MEMORY_MMAP;
            buf.index = i;
            if (-1 == xioctl(fd, VIDIOC_QBUF, &buf)) {
                perror("Queue Buffer");
                return 1;
            }
            printf("12345\r\n");
        }
        if (-1 == xioctl(fd, VIDIOC_STREAMON, &buf.type)) {
            perror("Start Capture");
            return 1;
        }
        return 0;
    }
    void release_device(int fd)
    {
        int type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
        xioctl(fd, VIDIOC_STREAMOFF, &type);
        close(fd);
    }
    gboolean bus_call(GstBus *bus, GstMessage *msg, gpointer data)
    {
        GMainLoop *loop = (GMainLoop *) data;
        switch (GST_MESSAGE_TYPE (msg))
        {
        case GST_MESSAGE_EOS:
            fprintf(stderr, "End of stream\n");
            g_main_loop_quit(loop);
            break;
        case GST_MESSAGE_ERROR:
        {
            gchar *debug;
            GError *error;
            gst_message_parse_error(msg, &error, &debug);
            g_free(debug);
            g_printerr("Error: %s\n", error->message);
            g_error_free(error);
            g_main_loop_quit(loop);
            break;
        }
        default:
            break;
        }
        return TRUE;
    }
    int main(int argc, char **argv)
    {
        App *app = &s_app;
        printf("==========\n");
        char devnode[100] = "/dev/video1";
        fd = open(devnode, O_RDWR);
        if (fd == -1) {
            perror("Opening video device");
            return 1;
        }
        ret = init_device(fd);
        if (0 != ret) {
            printf("Exiting");
            return ret;
        }
        gst_init(NULL,NULL);
        app->pipeline = gst_pipeline_new("encodepipeline");
        g_assert(app->pipeline);
        app->appsrc = gst_element_factory_make("appsrc","srcElement");
        g_assert(app->appsrc);
        app->encode = gst_element_factory_make("ducatih264enc","encodeElement");
        g_assert(app->encode);
        app->parse = gst_element_factory_make("h264parse","parseElement");
        g_assert(app->parse);
        app->decode = gst_element_factory_make("ducatih264dec","decodeElement");
        g_assert(app->decode);
        app->sink = gst_element_factory_make("filesink","sinkElement");
        g_assert(app->sink);
        printf("element creat success\n");
        GstCaps *capsappsrc2H264enc;
        capsappsrc2H264enc = gst_caps_new_simple("video/x-raw",
                                                 "format",G_TYPE_STRING, "NV12",
                                                 "width", G_TYPE_INT, 352,
                                                 "height",G_TYPE_INT, 288,
                                                 "framerate", GST_TYPE_FRACTION, 30, 1,
                                                 NULL);
    //    g_object_set(G_OBJECT (app->sink), "sync", FALSE,
    //                 NULL);
        gst_app_src_set_caps(GST_APP_SRC(app->appsrc), capsappsrc2H264enc);
        g_object_set(app->sink,"location","/mnt/camera.yuv",NULL);
        app->loop = g_main_loop_new(NULL,FALSE);
        app->bus = gst_pipeline_get_bus(GST_PIPELINE(app->pipeline));
        app->bus_watch_id = gst_bus_add_watch(app->bus,bus_call,app->loop);
        gst_object_unref(app->bus);
        gst_bin_add_many(GST_BIN(app->pipeline), app->appsrc,app->sink,NULL);
        gboolean bLinkOk;
        bLinkOk= gst_element_link(app->appsrc,app->sink);
        if(!bLinkOk){
            g_warning("Failed to link many 1\n");
            return -5;
        }
        g_signal_connect(app->appsrc, "need-data", G_CALLBACK(feed_data), app);
        gst_element_set_state (app->pipeline, GST_STATE_PLAYING);
        printf("run....1\n");
        g_main_loop_run(app->loop);
        printf("run....2\n");
        app->msg = gst_bus_timed_pop_filtered (app->bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
        if (app->msg != NULL)
            gst_message_unref (app->msg);
        gst_object_unref (app->bus);
        gst_element_set_state (app->pipeline, GST_STATE_NULL);
        gst_object_unref (app->pipeline);
        gst_object_unref(app->appsrc);
        gst_object_unref(app->sink);
        printf("close...\n");
        return 0;
    }

  • 0 in reply to abayyy
    TI__Guru* 80190 points
    Hello,

    So the memory leak is observed in the application only?
    Is it observed the same application but when fakesink is used or only in case of filesink?
    I will take a look into the application.
    One more question, you are capture via v4l2src right, what is the reason to use appsrc element instead of v4l2src element?

    BR
    Margarita
  • 0 in reply to Margarita Gashova
    TI__Guru* 80190 points
    Hello,

    Actually I see that in the application you have encode->decode.
    Could you remove the decode part or you already did this test?
    I am sorry for all these questions but I want to understand the use case.

    BR
    Margarita
  • 0 in reply to Margarita Gashova
    Intellectual 300 points
    I am so sorry, the gst-launch-1.0 -v -e v4l2src device=/dev/video1 io-mode=2 ! 'video/x-raw, format=(string)NV12, width=(int)352, height=(int)288, framerate=(fraction)30/1,bitrate=(int)30' ! ducatih264enc ! filesink location=test.h264 is also memory leak
  • 0 in reply to Margarita Gashova
    Intellectual 300 points
    yes ， remove the decode element is also memory leak
  • 0 in reply to abayyy
    TI__Guru* 80190 points

    Hello,

    Do you observe it with videnc2test also?

    BR
    Margarita

  • 0 in reply to Margarita Gashova
    Intellectual 300 points

    whit is videnc2test? 

    when i use it I need a large enough file for encode? loop encode it?

  • 0 in reply to abayyy
    TI__Mastermind 22946 points
    We will try to dig into this memory leak issue around mid September and get back with our findings.
  • 0 in reply to Manisha Agrawal
    TI__Mastermind 30480 points

    Hi,

    Is this issue not reproduced with ducatimpeg4enc?

    Please share me what memory statistics you are collecting to narrow down ducatih264enc as the root cause of the issue.

  • 0 in reply to Ramprasad
    Intellectual 300 points

    I haven't used ducatimpeg4enc, about ducatih264enc you can Look at this page e2e.ti.com/.../2287603

    I only use top command for watch memory leak . 

  • 0 in reply to abayyy
    TI__Mastermind 22946 points

    Hi Abayyy,

    The encoder memory leak issue has been fixed and the patch is available here - 

    You can use arago build to rebuild the plugin -

  • 0 in reply to Manisha Agrawal
    Intellectual 300 points

    Hi,manisha,

    thanks,but I don't know how to only build the ducatih264enc plugin with ti-processor-sdk-linux-am57xx-evm-03.03.00.04 SDK? Can you help me?

  • 0 in reply to abayyy
    TI__Guru* 80190 points

    Hello,

    abayyy said:
    thanks,but I don't know how to only build the ducatih264enc plugin with ti-processor-sdk-linux-am57xx-evm-03.03.00.04 SDK?

    The issue was not observed only for ducatih264enc.


    This is the user guide for PSDK version 03.03.00.04
    processors.wiki.ti.com/index.php

    Please follow the steps in the guide.
    This is the last step for building ducati plugin:
    MACHINE=am57xx-evm bitbake gstreamer1.0-plugins-ducati

    You could apply the patch manually in the gstducatividenc.c file.
    Please check chapter Forced Re-compilation in the guide.
    The new libgstducati.so file should be cp to rootfs/usr/lib/gstreamer-1.0.

    Hope this helps.

    BR
    Margarita

  • 0 in reply to Margarita Gashova
    Intellectual 300 points
    thanks,Margarita
    In China, using the Internet is very slow and often disconnected, Can you provide me with a compiled one? My SDK is 03.03.00.04.
  • 0 in reply to abayyy
    TI__Guru* 80190 points
    Hello,

    Unfortunately I do not have 3.03.00.04 on my side.

    BR
    Margarita
  • 0 in reply to Margarita Gashova
    TI__Intellectual 1930 points

    Hi Margarita,

    Which version of PDK that your patch of duchati plugin work on? I think the .so file does not depend on the PDK version.

    Thanks for your great support!

  • 0 in reply to Jian Zhou
    TI__Intellectual 1930 points
    Hi Margarita,
    As it needs time for customer to recompile the Gstreamer plugin .so file, could you please directly send the .so file with patch for customer's quick verification? Thanks so much!
  • 0 in reply to Jian Zhou
    TI__Guru* 80190 points

    Hello,

    Check the attachment:

    libgstducati.zip

    BR
    Margarita

  • 0 in reply to Margarita Gashova
    Intellectual 300 points


    Thank you very much for your help to solve the memory leak problem

    https://e2e.ti.com/support/embedded/linux/f/354/p/620539/2291680#2291680

  • 0 in reply to Margarita Gashova
    Prodigy 200 points
    Hi Margarita,

    We are using "3.2.0.5" SDK release of AM57XX, and facing memory leak in "ducatih264enc".
    By using attached "libgstducati.so" we have seen that issue is resolved.
    We have compiled "libducati.so" by using yocto build with our customized changes, Is it possible to get source code of this ".SO", So that we can avoid entire yocto build setup.

    Regards,
    Nirav
  • 0 in reply to Nirav Patel
    TI__Mastermind 30480 points
    Hi Nirav,
    git.ti.com/.../7205d7cb75944e816622d5daca88804819ff16d3
    is the change required for this fix. This is one line change.

    You can clone entire repository from git://git.ti.com/glsdk/gst-plugin-ducati.git

    Ramprasad
  • 0 in reply to Ramprasad
    TI__Intellectual 1930 points

    Hi Margarita,

    Customer do not find the memory leak issue in the H264enc plugin by the .so file you provided, thanks for your support!

    Could you please also check the ducatih264dec plugin, customer reported that there is also memory leak issue in this plugin, thanks!

  • 0 in reply to Jian Zhou
    TI__Guru* 80190 points

    Hello,

    I will check.
    The issue is observed with gstreamer pipeline not an application, right?



    BR
    Margarita

  • 0 in reply to Margarita Gashova
    TI__Guru* 80190 points

    Hello,

    We are able to reproduce the issue.

    BR
    Margarita

  • 0 in reply to Margarita Gashova
    TI__Intellectual 1930 points

    Hi Margarita:

    Customer's test scenario is as follows:

    gst-launch-1.0 -v -e v4l2src device=/dev/video1 io-mode=2 ! 'video/x-raw, format=(string)NV12, width=(int)800, height=(int)600, framerate=(fraction)30/1,bitrate=(int)30' ! ducatih264enc ! h264parse ! ducatih264dec ! waylandsink sync=false

     

    So you can reproduce this issue? And could you please provide the updated plugin after the memory leak issue is fixed? Thanks

  • 0 in reply to Jian Zhou
    TI__Guru* 80190 points
    Hello,

    Could you try to replace ducatih264dec with avdec_h264 and run the test for an hour. Do you observe the mem leak issue on your side?
    Note: please make sure that the .so for encoder mem leak fix is applied.

    We tried on our side this pipeline:
    gst-launch-1.0 -e videotestsrc ! 'video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! ducatih264enc ! queue ! h264parse ! avdec_h264 ! videoconvert ! kmssink &
    but on our side the leak was observed even with avdec_h264.

    We tried this pipeline also:
    gst-launch-1.0 -e videotestsrc ! 'video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! ducatih264enc ! queue ! h264parse ! 'video/x-h264, stream-format=byte-stream' ! h264parse ! ducatih264dec ! kmssink &

    We are trying to narrow down the leak.



    BR
    Margarita
  • 0 in reply to Margarita Gashova
    TI__Mastermind 30480 points

    Hi Jian,

    I reproduced the issue with your pipeline.  I analyzed the ducatidecoder source code, I found one small memory leak every frame but after fixing this leak also still observe Mem% in top command increasing .

    I have attached the modified libgstducati.so for your testing. Can you please confirm the behaviour?

    One more observation is if kmsink in the pipeline is replaced with fakesink then no memoryleak is obsrved.

    Issue is seen with kmssink or waylandsink.

    6012.libgstducati.zip

    Ram

  • 0 in reply to Ramprasad
    TI__Intellectual 1930 points

    Hi Ram,

    Thanks a lot for your support and hard work!

    So you have fixed the memory leak issue in the ducatidecoder plugin and also analyzing the code in kmssink or waylandsink?

    I will work with customer to test based on your guidance, thanks!

  • 0 in reply to Jian Zhou
    TI__Mastermind 22946 points
    Hi Jian,

    We did preliminary analysis on kmssink and waylandsink and do not see the memory leak with those plugins.

    We look forward to feedback from customer after using the memory leak fixed plugins. If further issues are with plugins that's opensource and not provided by TI, I am afraid, we won't be able to help much on them.
  • 0 in reply to Manisha Agrawal
    TI__Intellectual 1930 points

    Manisha,

    Thanks for your support! I will work with customer to verify after the China National Holiday!