TDA4VM: The camera's encoded video stream is experiencing jitter. May I ask what may be the reason for this?

Hello,

Sorry I was out of office these past couple of days. I will look into this and let you know. Are you able to measure the CPU utilization here?

BR,
Sarabesh S.

Hi，

The current CPU usage is relatively high, with two cores accounting for approximately 150%. However, during our early testing process, there were instances of excessive CPU usage, and we did not find this issue.

Regards,

Yq

Hello, 

Since there is excessive CPU usage it could be the case that buffer overflow is causing the jitter. Could you share the GStreamer pipeline that is running in the app_multi_cam_codec demo?

Thanks,
Sarabesh S.

Hi，

Will the use of hardware encoding and decoding methods also be affected by the CPU? This is information related to the GStreamer pipeline：

snprintf(params->m_AppSrcNameArr[ch] , CODEC_MAX_LEN_ELEM_NAME, "myAppSrc%d" , ch);
            i += snprintf(&params->m_cmdString[i], CODEC_MAX_LEN_CMD_STR-i,"appsrc format=GST_FORMAT_TIME is-live=true do-timestamp=true block=false name=%s ! queue \n",params->m_AppSrcNameArr[ch]);
            i += snprintf(&params->m_cmdString[i], CODEC_MAX_LEN_CMD_STR-i,"! video/x-raw, width=(int)%d, height=(int)%d, framerate=(fraction)30/1, format=(string)%s, interlace-mode=(string)progressive, colorimetry=(string)bt601 \n",
                                                                                            params->in_width, params->in_height, params->in_format);
            i += snprintf(&params->m_cmdString[i], CODEC_MAX_LEN_CMD_STR-i,"! v4l2h264enc io-mode=dmabuf-import bitrate=10000000 \n");

Regards，

Yueqian

Hi Yq,

Is there a decode->display GStreamer portion of the demo app that you can share?

Thanks,
Sarabesh S.

Hi,

I did not use 'a decode ->display GStreamer portion of the demo app', but instead used 'multifilesink' to write data to the H264 file.

Additionally, I encoded four videos simultaneously, but not every one had this phenomenon.

Regards，

Yueqian

Hi Yq,

Are you writing to a file through the demo app? I don't see the multifilesink element in the demo app pipeline you provided. Would you be able to share the compressed h264 streams that you saved with jitter and without?

BR,
Sarabesh S.

Hi Yq, 

Sounds good. This thread will lock after 30 days. If that's the case please open a new one.

BR,
Sarabesh S.

OK, thank you!

Regards

Yueqian

Hi，

We have encountered this problem again, even though the system CPU usage is low at this time, there will still be frequent video jitter.

The data is obtained from the buffer and written to the file in appGstDeqAppSink through the following method.：

“p_gst_pipe_obj->pulled_buff[idx][ch] = gst_sample_get_buffer(out_sample);

gst_buffer_map(p_gst_pipe_obj->pulled_buff[idx][ch], &info, (GstMapFlags)(GST_MAP_READ));”

And this phenomenon does not always occur, sometimes it is good, and sometimes the shaking is particularly severe.

This phenomenon is not necessary, so how can we investigate whether it is a hardware problem or a software problem? On some hardware, this issue has never occurred and the same software is used.

Regards，

Yueqian

Hello,

What hardware are you using? Is it the TI EVM that you are seeing this problem on?

Thanks,
Sarabesh S.

Hi,

The chip we use is TDA4VM. 

Regards，

Yueqian

Hello,

I understand but is this a problem on the TI EVM or is the SOC on another hardware platform being used? I will raise this internally to see if this is a VPU, GPU or something else causing this. 

Thanks,
Sarabesh S.

Hi,

We are applying the TDA4VM chip to other hardware devices.

Regards,

Yueqian

Hi Yueqian,

In that case, do you know if the problem could be with the other hardware that the TDA4VM is integrated with?

Thanks,
Sarabesh S.

Hi,

Currently, based on hardware signal measurements, there are no abnormalities. But I can't be sure if it's not a hardware issue, so I want to investigate whether it's caused by a software problem? Moreover, among the four channels of video, only the first channel is particularly severe, while the second channel is relatively better. The third and fourth channels of video do not have this problem. If it is not caused by software issues, we will focus on investigating hardware problems

Regards,

Yueqian

Hi Yueqian, 

I don't know of any known software problems that cause this "jitter" than you are seeing. From looking at the video it seems like a rendering problem, possibly GPU related. 

Just to be clear not all of the tda4vm h/w and s/w are having this problem and only 2/4 channels of the individual platforms are experiencing this?

Thanks,
Sarabesh S.

Hi,

Yes, the first channel is more severe.

If it is a GPU rendering problem, is there any way to solve or improve it? Will GPU problems cause only one or two channels to experience this phenomenon?

Regards，

Yueqian

Yueqian,

The multi-cam-codec doesn't directly use the GPU module in the pipeline by default. By any chance did you make any modifications to use the GPU at all in your usecase?

Thanks,
Sarabesh

Hi Sarabesh,

Does' app_multi_cam_codec 'only use CPU? I did not actively modify the code to use GPU. Currently, the process is consuming a relatively large amount of CPU. If there is a way to use GPU and other modules to reduce CPU usage, it would be a great improvement.

Regards

Yueqian

Hi Yueqian, 

We do not have a known nor published way of offloading processing to the GPU for this use-case. There are only a couple of demos that utilize the GPU in Vision-Apps such as the surround view demo. 

There may be a way to debug whether this is a hardware or software problem using a open_vx capture function to capture the buffer at each point of the pipeline to determine where the screen tear is occurring.

Thanks,
Sarabesh S. 

Hi,

May I ask how to use a 'open-vx' capture function to capture the buffer at each point of the pipeline to determine where the screen tear is occurring?

Do you mean to view every frame image of each node? I guess this is a problem with encoding between frames? It's not a problem with single frame images.

Regards，

Yueqian

I will have to discuss internally with our openvx-expert to understand how to use this function. I will get back to you early next week on this.

qian yue said:

Do you mean to view every frame image of each node? I guess this is a problem with encoding between frames? It's not a problem with single frame images.

There is not a known problem with our hardware or software for encoding.

BR,
Sarabesh S.

OK, looking forward to your reply.

Regards,

Yueqian

Sounds good, I will let you know.

Sarabesh S.

Hi Yueqian, 

Thank for your patience on this. I was looking to implement this but got occupied with other escalations. Were you able to deduce what was causing the screen tear whether it was software or hardware?

BR,
Sarabesh

Hi,

I haven't located the reason yet.

Regards,

Yueqian