This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

v4l2 gstreamer pipe crashes under load

Hi All

I'm using EZSDK_5_05 (NOTE: the problem occurs on 5_04 as well) and the dm8168 processor. We have our own hardware with an FPGA that converts SDI video to BT1120 format that comes into the vin0 port. We've modified the vin drivers to support this configuration and 720p50 video and have the following gstreamer pipe running:

"gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=6 ! video/x-raw-yuv-strided, format=(fourcc)NV12, framerate=(fraction)50/1, width=(int)1280, height=(int)720 ! omxbufferalloc numBuffers=6 ! omx_h264enc profile=8 level=2048 force-idr=true force-idr-period=25 i-period=0 bitrate=12000000 rateControlPreset=0 input-buffers=6 output-buffers=8 ! queue ! rtph264pay ! udpsink host=224.0.1.17 ttl-mc=32 port=5004 -e -v --gst-debug=v4l2src:2"

This pipe continues to run without issues for 5+hours and then suddenly stop without any error messages. Its as if the video source dried up. However, I've managed to force this condition to happen after 30 seconds by supplying a video source thats got a random noise pattern. However, if I remove the "rtph264pay" transport plugin from the pipe it continues running even with the noise pattern and even if I load the processor to 100% with a test app with a while(1) loop. Replacing rtph264pay with mpegtsmux transport stream forces the issue to happen immediately with the noise pattern.

Can anyone please give advice how to pin-point/debug this type of issue, at the momemt I'm not sure is the problem in gstreamer, gstreamer plugins, omx, device drivers or even in the M3 code?

Kind regards

Johan

  • Hello,

    You could debug the gstreamer with --gst-debug.

    You could use LoggerSMDump Utility, it will provide you debug logs generated by the Media Controller.

    http://processors.wiki.ti.com/index.php/OMX_Viewing_Media_Controller_Traces

    ./loggerSMDump.out 0x9E400000 0x100000 all

    Hope this to help you.

    Best Regards,

    Margarita

  • Hi Margarita

    Thanks for the above, I've enabled that before without any significant info; e.g. it was as if the video input just stopped. How can I enable the debug messages of the vin driver; e.g. ti81xxvin_main.c

     

    Regards

    Johan 

  • Hello,

    You could check here:

    http://processors.wiki.ti.com/index.php/TI81XX_PSP_VIDEO_CAPTURE_Driver_User_Guide

    Command Line arguments

    Best Regards,

    Margarita

  • Hi Margarita

    I've enabled the debug (please see trace below) and let it run for a while with normal video, then I've swithed to a noise video pattern to force the issue that I'm having. You can see the trace stops with the last method called being ti81xxvin ti81xxvin: ti81xxvin_buffer_queue.

    Do you have any ideas how to progress further, steps I can take to dig a bit deeper? Is the problem in ti81xxvin_buffer_queue?

    ...

    ti81xxvin ti81xxvin: ti81xxvin_buffer_prepare

    ti81xxvin ti81xxvin: ti81xxvin_buffer_queue

    ti81xxvin ti81xxvin: vidioc_dqbuf

    ti81xxvin ti81xxvin: vidioc_qbuf

    ti81xxvin ti81xxvin: vidioc_qbuf

    ti81xxvin ti81xxvin: ti81xxvin_buffer_prepare

    ti81xxvin ti81xxvin: ti81xxvin_buffer_queue

    ti81xxvin ti81xxvin: vidioc_dqbuf

    ti81xxvin ti81xxvin: vidioc_qbuf

    ti81xxvin ti81xxvin: vidioc_qbuf

    ti81xxvin ti81xxvin: ti81xxvin_buffer_prepare

    ti81xxvin ti81xxvin: ti81xxvin_buffer_queue

    ti81xxvin ti81xxvin: vidioc_dqbuf

    ti81xxvin ti81xxvin: vidioc_qbuf

    ti81xxvin ti81xxvin: vidioc_qbuf

    ti81xxvin ti81xxvin: ti81xxvin_buffer_prepare

    ti81xxvin ti81xxvin: ti81xxvin_buffer_queue

  • Hi Margarita

    There are numerous bug fixes in the ti81xxvin_main.c driver file which allows me to run the video under load (e.g. Noise Video pattern) without it freezing. I suggest that the owner of the file fixes them for the next EZSDK release and investigate how a live-lock condition can happen around the vidioc_qbuf & vidioc_dqbuf functions.

    Fixes:

    vidioc_querybuf(): returning without unlocking mutex under certain conditions

    vidioc_streamon(): unlocking mutex twice

    vidioc_streamoff(): returning without unlocking mutex under certain conditions

    ti81xxvin_mmap(): returning without unlocking mutex under certain conditions

    ti81xxvin_overflow_handler(): returning without unlocking buf_irqlock under certain conditions

    Load issue:
    vidioc_qbuf & vidioc_dqbuf: under load there is a live-lock situation around the mutexes (for some reason it stops at mutex when trying to queue buffers), for the moment I've removed the mutex locking and unlocking in these two functions.

  • Hello,

    I am glad that this issue is solved, for now.

    Best Regards,

    Margarita