Video capture jitter problems with DM355

Hey Nick,

I believe I've seen the same problem.  Even though I mentioned before I'm getting the correct frame rate out of the encoder, the video does not appear to be smooth.  I knew I had the correct frame, but couldn't explain the jitter.  This is 720x480 resolution encoded into h.264.

John

Hi Nick,

Are you using capture driver in chained mode where ipipe+resizer is running in continuous mode OR are you running ipipe+resizer in single shot mode?

If you are running in singleshot mode, then i would recommend you to reduce the input clock to the resizer by increasing the clock divider. You can get this in RSZ_S_CONFIG ioctl. It might be possible that the resizer is not completing and the old captured frame is coming back for MPEG4 processing.

Ofcourse the other potential problem can be the buffer switching logic as you have mentioned. Have you tried to look at buffer logic in the user space and is that causing any issues?

Regards,

Anshuman

PS: Please mark this post as verified, if you think it has answered your question. Thanks.

I believe I am running the IPIPE in single shot mode. I have a sensor configured to produce 1280x720 data (raw sensor data). This data is then fed into the IPIPE and NOT resized. Currently I have the ipipeif_param.clk_div set to DIVIDE_THIRD. If I set it to DIVIDE_HALF I get tearing and glitches on the image. I can go slower, but then I get a different type of tearing in the image.

Once the IPIPE has produce a frame I buffer the data (different code and buffer than the V4L2 architecture) and send it to the MPEG encoder. Does the MPEG encoder have an understanding of time? For example I buffer images prior to sending them to the MPEG encoder because from time to time something runs in Linux that would cause me to drop a frame. But if I buffer the frames, on average, the MPEG encoder can keep up.

As I mentioned before, I mocked a system where I fed two fixed images into the MPEG encoder. I injected the frames at the same point where the code normally takes frames from the IPIPE and puts them into the buffer. When I did this I did not see any glitches. I tried to repeat the same experiment, but this time inject frames immediately before I send the RAW frames into the IPIPE. This didn't work and I haven't been able to find out why.

What it appears to me is that one of the following is happening:

1) When the vpfe_isr fires in the davinci_vpfe.c file, it updates the current buffer and calls wake_up_interruptible:

<code>

    } else if (frm_format == CCDC_FRMFMT_PROGRESSIVE) {

        uint32_t now = davinci_readl(DAVINCI_TIMER0_BASE + TIM34);

        //printk("vpfe->curFrm = %08X  vpfe->nextFrm = %08X  time = %u\n", vpfe->curFrm, vpfe->nextFrm, now);

        dev_dbg(vpfe_dev, "\nframe format is progressive...");
        if (vpfe->curFrm != vpfe->nextFrm) {
            /* Copy frame capture time value in curFrm->ts */
            vpfe->curFrm->ts = timevalue;
            vpfe->curFrm->state = STATE_DONE;
            wake_up_interruptible(&vpfe->curFrm->done);
            printk("vpfe_isr: current frame = %08X\n", vpfe->curFrm);
            vpfe->curFrm = vpfe->nextFrm;
        }
        else
        {
            printk("vpfe losts a frame\n");
        }
</code>

From time to time, I suspect this returns the same buffer. So I print out vpfe lost a frame when this happens. But this happens rarely. Not nearly as frequent as the double frames occur in the video stream

2) There is some issue with the way the CCDC writes data and the way the IPIPE pulls the data in

Here we are dealing with two black boxes that are poorly documented. So I really have no way to verify if this is happening

3) The MPEG encoder produces two output frames from the same input frame

The two frames are compressed differently. They have different motion vectors. So it "looks" like the MPEG encoder saw 2 frames. I'm starting to think this is not happening. I can see the CCDC produced X frames and the video stream has Y frames, where Y = X - 2. Which is possible, because the frames are produced and put in a buffer. So it is possible the software was shut down before the MPEG encoder had time to consume those frames. If the MPEG encoder produced additional frames, then I would expect to see Y > X, which is never the case.

Summary

At this point I think it is something to do with the CCDC module or the V4L2 buffer logic. Any ideas from the community would be GREATFULLY appreciated.

Here are images 6, 7 and 8 from the small clip I posted above. Images 6 and 7 are clearly different, but image 8 is the same as image 7.

Image 6

Image 7

asdf

asdf

I posted the images here:

http://sites.google.com/site/nlbutts/

Hi Nick,

I cant see the images that you uploaded. Actually, seeing the images or videos would surely help to understand the problem.

BTW, i would like to try following things:

1. Tell me what is the VDINT0 and VDINT1 value and what is the width of your frame?

2. Can you eliminate the MPEG4 encoder by removing it completely and just connecting the IPIPE output to display. This way we can confirm if the issue is in the IPIPE output itself or not.

3. Can you re-look at the buffer logic between the CCDC output and the IPIPE input?

4. Can you please write some dummy data (say 0xABCD1234) at the last 4 pixels of the output frame of IPIPE and check that after every IPIPE operation is it getting modified or not? If it is not getting modified, it might be possible that IPIPE (resizer output port) is not able to keep up with the input data frequency.

5. Can you please share images with the different tearing effect when you change the clk_div settings.

Regards,

Anshuman

Anshuman Saxena said:

1. Tell me what is the VDINT0 and VDINT1 value and what is the width of your frame?

Anshuman Saxena said:

2. Can you eliminate the MPEG4 encoder by removing it completely and just connecting the IPIPE output to display. This way we can confirm if the issue is in the IPIPE output itself or not.

Anshuman Saxena said:

3. Can you re-look at the buffer logic between the CCDC output and the IPIPE input?

Anshuman Saxena said:

4. Can you please write some dummy data (say 0xABCD1234) at the last 4 pixels of the output frame of IPIPE and check that after every IPIPE operation is it getting modified or not? If it is not getting modified, it might be possible that IPIPE (resizer output port) is not able to keep up with the input data frequency.

Anshuman Saxena said:

5. Can you please share images with the different tearing effect when you change the clk_div settings.

I have tried to implement something so I can get the CCDC frames written to SD, but that takes far too long to write to be useful.

I just implemented number 4. I write 0x55AA55AA to the last 4 bytes of a frame. I then tell the IPIPE to execute. I then check to see if the buffer is still 0x55AA55AA. So far it has always overwritten the data. I can time how long it takes to run through the IPIPE.

Attached is an Excel sheet that shows timing information on how long it takes to capture an image, run it through the IPIPE and compress it. Here is a summary of the data:

7444.MPEG pipeline.xlsx

Hi Nick,

Nick Butts said:

I've tried two different settings. In ccdc_dm355.c, the function ccdc_setwin sets VDINT0 to image_win->top which is 27 and VDINT1 is set to VDINT0 + (image_win->height / 2). My image size is 1280x720, so it gets set to 387. Although later in ccdc_config_raw sets it to some magic numbers:

VDINT0 = 136

VDINT1 = 149

I've tried both settings and they don't appear to make any difference.

I would suggest that you put VDINT0 and VDINT1 to something around 640. You can override what linux kernel is writing.

Meanwhile, i will look into all the data that you have shared.

Regards,

Anshuman

Hi Nick,

Good to know that you could find the problem and solve it. Thanks for sharing the results

Regards,

Anshuman

Hi Anshuman,

I am also facing the similar Image tearing issue.Please find the attached image.

I tried changing the    params.ipipeif_param.clk_div          = DIVIDE_SIXTH

to DIVIDE_THIRD but i was getting the same image tearing issue.

Please also find the dump of IPIPE and RESIZER registers below:

# ./regrw.out 1c70800 100 -c 1
 
 0x01c70800: 00000001
 0x01c70804: 00000000
 0x01c70808: 00000000
 0x01c7080c: 0000004e
 0x01c70810: 00000012
 0x01c70814: 0000043f
 0x01c70818: 00000034
 0x01c7081c: 0000065f
 0x01c70820: 00000000
 0x01c70824: 00000004
 0x01c70828: 00000001
 0x01c7082c: 0000000f
 0x01c70830: 00000000
 0x01c70834: 00000001
 0x01c70838: 00000003
 0x01c7083c: 00000000
 0x01c70840: 000000ff
 0x01c70844: 00000001
 0x01c70848: 00000001
 0x01c7084c: 00000100
 0x01c70850: 00000100
 0x01c70854: 00000100
 0x01c70858: 00000100
 0x01c7085c: 00000080
 0x01c70860: 00000080
 0x01c70864: 00000080
 0x01c70868: 00000080
 0x01c7086c: 00000003
 0x01c70870: 00000040
 0x01c70874: 00000001
 0x01c70878: 00000010
 0x01c7087c: 00000080
 0x01c70880: 00000000
 0x01c70884: 00000000
 0x01c70888: 00000000
 0x01c7088c: 00000000
 0x01c70890: 00000000
 0x01c70894: 00000000
 0x01c70898: 00000000
 0x01c7089c: 00000000
 0x01c708a0: 00000000
 0x01c708a4: 00000000
 0x01c708a8: 00000000
 0x01c708ac: 00000000
 0x01c708b0: 00000000
 0x01c708b4: 00000000
 0x01c708b8: 00000000
 0x01c708bc: 00000000
 0x01c708c0: 00000000
 0x01c708c4: 00000000
 0x01c708c8: 00000000
 0x01c708cc: 00000000
 0x01c708d0: 00000000
 0x01c708d4: 00000000
 0x01c708d8: 00000001
 0x01c708dc: 00000010
 0x01c708e0: 00000021
 0x01c708e4: 0000001f
 0x01c708e8: 0000001d
 0x01c708ec: 0000001a
 0x01c708f0: 00000018
 0x01c708f4: 00000016
 0x01c708f8: 00000014
 0x01c708fc: 00000012
 0x01c70900: 00000019
 0x01c70904: 00000016
 0x01c70908: 00000014
 0x01c7090c: 00000012
 0x01c70910: 00000011
 0x01c70914: 00000011
 0x01c70918: 00000010
 0x01c7091c: 00000010
 0x01c70920: 00000000
 0x01c70924: 00000000
 0x01c70928: 00000000
 0x01c7092c: 00000000
 0x01c70930: 00000000
 0x01c70934: 00000000
 0x01c70938: 00000000
 0x01c7093c: 00000000
 0x01c70940: 00000014
 0x01c70944: 00000258
 0x01c70948: 00000000
 0x01c7094c: 00000003
 0x01c70950: 00000017
 0x01c70954: 00000021
 0x01c70958: 0000002c
 0x01c7095c: 00000036
 0x01c70960: 0000003f
 0x01c70964: 00000048
 0x01c70968: 00000051
 0x01c7096c: 0000005a
 0x01c70970: 00000010
 0x01c70974: 0000000f
 0x01c70978: 0000000f
 0x01c7097c: 0000000e
 0x01c70980: 0000000d
 0x01c70984: 0000000c
 0x01c70988: 0000000b
 0x01c7098c: 0000000a





# ./regrw.out 1c70800 100 -c 1
 
 0x01c70800: 00000001
 0x01c70804: 00000000
 0x01c70808: 00000000
 0x01c7080c: 0000004e
 0x01c70810: 00000012
 0x01c70814: 0000043f
 0x01c70818: 00000034
 0x01c7081c: 0000065f
 0x01c70820: 00000000
 0x01c70824: 00000004
 0x01c70828: 00000001
 0x01c7082c: 0000000f
 0x01c70830: 00000000
 0x01c70834: 00000001
 0x01c70838: 00000003
 0x01c7083c: 00000000
 0x01c70840: 000000ff
 0x01c70844: 00000001
 0x01c70848: 00000001
 0x01c7084c: 00000100
 0x01c70850: 00000100
 0x01c70854: 00000100
 0x01c70858: 00000100
 0x01c7085c: 00000080
 0x01c70860: 00000080
 0x01c70864: 00000080
 0x01c70868: 00000080
 0x01c7086c: 00000003
 0x01c70870: 00000040
 0x01c70874: 00000001
 0x01c70878: 00000010
 0x01c7087c: 00000080
 0x01c70880: 00000000
 0x01c70884: 00000000
 0x01c70888: 00000000
 0x01c7088c: 00000000
 0x01c70890: 00000000
 0x01c70894: 00000000
 0x01c70898: 00000000
 0x01c7089c: 00000000
 0x01c708a0: 00000000
 0x01c708a4: 00000000
 0x01c708a8: 00000000
 0x01c708ac: 00000000
 0x01c708b0: 00000000
 0x01c708b4: 00000000
 0x01c708b8: 00000000
 0x01c708bc: 00000000
 0x01c708c0: 00000000
 0x01c708c4: 00000000
 0x01c708c8: 00000000
 0x01c708cc: 00000000
 0x01c708d0: 00000000
 0x01c708d4: 00000000
 0x01c708d8: 00000001
 0x01c708dc: 00000010
 0x01c708e0: 00000021
 0x01c708e4: 0000001f
 0x01c708e8: 0000001d
 0x01c708ec: 0000001a
 0x01c708f0: 00000018
 0x01c708f4: 00000016
 0x01c708f8: 00000014
 0x01c708fc: 00000012
 0x01c70900: 00000019
 0x01c70904: 00000016
 0x01c70908: 00000014
 0x01c7090c: 00000012
 0x01c70910: 00000011
 0x01c70914: 00000011
 0x01c70918: 00000010
 0x01c7091c: 00000010
 0x01c70920: 00000000
 0x01c70924: 00000000
 0x01c70928: 00000000
 0x01c7092c: 00000000
 0x01c70930: 00000000
 0x01c70934: 00000000
 0x01c70938: 00000000
 0x01c7093c: 00000000
 0x01c70940: 00000014
 0x01c70944: 00000258
 0x01c70948: 00000000
 0x01c7094c: 00000003
 0x01c70950: 00000017
 0x01c70954: 00000021
 0x01c70958: 0000002c
 0x01c7095c: 00000036
 0x01c70960: 0000003f
 0x01c70964: 00000048
 0x01c70968: 00000051
 0x01c7096c: 0000005a
 0x01c70970: 00000010
 0x01c70974: 0000000f
 0x01c70978: 0000000f
 0x01c7097c: 0000000e
 0x01c70980: 0000000d
 0x01c70984: 0000000c
 0x01c70988: 0000000b
 0x01c7098c: 0000000a

You have suggested changing the VDINT0 and VDINT1 settings.i will try that and see if it leads to any positive results.

In one of the posts you have suggested to disable encoding an d directly copy the ipipe o/p to display.

Does it mean commenting the encode task,exaclty how can we do this.

In my case image tearing comes in ISIF_IN mode D1 resolution in every codec.

I tried changing the synchronization register of resizer but that also was of no help.

Could you suggest anything to focus in order to solve this issue ASAP.

Thanks and Regards,

Mayank

Hello Nick,

Though it has been long time back since yo would be of greatu actually solved the issue

but if you could recall and tell what exacltly you did in the code or any register settings in order

to solve the Image tearing issue,then it would be of great help.

I am also struggling with this issue for around a week now.

Regards,

Mayank

Perhaps Nick will provide further details, but my interpretation of his resolution is simply that he had a cheap/slow SD card and getting a faster card was what he needed to resolve the issue.