Is the H.264 decoder broke?

John,

I am not sure about the DVSDK support for interlaced decoding. Will check and get back.

But DM36x H.264 decoder is very much capable of decoding intrelaced content. You can try using the sample test app provided with the decoder release and check the stream. The sample test app can also help you to understand the way decoder buffers needs to be presented for intrelaced content.

regards

Yashwant

Quite frankly, I'm not sure that the problem isn't in the encoder.  When I watch the video on VLC the jittery motion isn't as clear as watching an NTSC monitor using the dm368 as the decoder.  It's like the encoder is mixing  the bottom field of frame n with the top field of frame n+1.

I'd like to feed an entire frame to the decoder as a single buffer but It doesn't act like I would expect.  I would expect the decoder to consume only the amount of bytes for the first field if it's going to return a Dmai_EFIRSTFIELD code.

John A

Hi John,

I am able to run decoder demo application provided with DVSDK for interlace streams, the application reads stream from file and displays output on TV. Video on TV is fine, there is no flip flop of fields. Can you provide your stream I will try here.

And are you seeing the issue only when you provide one frame/field data to decoder?

I am using DVSDK3.1.

I'm coming to the conclusion that the encoder is my problem.  I will provide a sample video file soon.

John A

Here is a sample h.264 encode that demonstrates the field order problem.  When you run it with the dvsdk sample decoder you can see the jitter trails on the moving cars.  If I encode the bottom field first the jitter goes away but the field order is visually reversed.

http://dl.dropbox.com/u/3345452/test.264

John A

Hi John,

We decoded the stream with Decoder demo, observed jitter trails on moving cars. And same jitters trials can be observed when we play stream in VLC player.

We open stream in stream analyser with Top field and bottom fieldd seperately, motion in bottom field is lagging compare to Top field. In Interlace video when we  encode top field 1st, motion in bottom field  will be ahead of top field but in this streams its not. So we suspect input to the encoder might going wrong. Please check input YUV.    

If your YUV is fine, please share your encoded stream with bottom field 1st.

Thanks for checking this Veeranna.  It's nice to confirm that it's in the encoder.  So now the question is whether it's in the UYVU or a problem with the encode operation.  I have to write a tool everytime I want to check something like this.  I've already wrote a simple app to render the UYVY and 420 frames.  Maybe I'll render the UYVY in gray scale and make the even line green hue and the odd ones red.  That way I can see if the issue is in the UYVY capture.

John A

I dont think the problem is with encoder. The problem seems to be in the way the encoder is getting the YUV input. The encoder is verified to be working fine with interlaced mode.

regards

Yashwant

I verified that the problem exists on the DM365EVM, so that eliminates the possibility that we made an error on our board design.

I really don't see what I could be doing to mess up the YUV input.  I'm using DMAI calls so it's all pretty high level, hands off the guts.

Does anyone have any interlaced video captured with the composite input of the EVM?  Maybe this design is untested.

John A

This is a frame of the UYVY data.  I have rendered the top field even lines (starting at 0) with green and the bottom field odd lines with red.  The top field is trailing behind the car, which means the bottom field is rendered second.  IOW the UYVY data looks correct in temporal order.  This leads back to the encoder not working correctly.

John A

Hi John,

In this image it looks that red lines are at top. To confirm encoder functionality, please run test app provided with package for interlace video. Test app has routines to convert 420 planar to 420 uv interleaved and uv interleaved to 420 planar. We tested it for many interlace inputs we did not see such issue. 

It looks like the top field is red because of the 5 lines of black video above the first red line.  So the 1st red line is line 5 (numbered 0..479).

I'm guessing that the test program is not the one using DMAI, which I am.  I'll take a look.

John A

The encode program in ths dvsdk -demos directory does not encode interlaced video.  The test apps under the codecs hierarchy do not build using the sdk top level make.  It's not worth trying to figure out how to compile these apps as it does nothing for me.  I'm using the DMAI API, so even if it encodes properly I wouldn't know what to change.

I will take a look at the code and see if any kind of parameters might be used that I'm not using.  That would be the only help it would provide.

My theory right now is that the bottom field of frame N is being encoded with the top field of frame N+1.  If I'm right I could play some tricks with what I send to the encoder.  If that works then I go with that and let those who follow figure it out themselves and use my posts as a clue.

John A

John,

I just saw the test.264 which you have shared. I feel that is encoded fine. I see the top field ahead which should be if top field is captured first. The car edge appears jittery to me because it is an interlaced content and I am playing it on a progressive display (my PC/laptop). Had it been played on a TV, it would have looked fine. Where are you seeing the o/p after decoder ?

regards

Yashwant

Yashwant,

I agree the top field is ahead, but should not be because it's captured first.  The bottom field captured last is later in time, and since the car has advanced forward the latest field should be ahead.

I am viewing this video using the DM368 composite output displayed on an NTSC CRT monitor.  The jitter is present and clearly indicates a problem with the video.

John A

John Anderson said:

My theory right now is that the bottom field of frame N is being encoded with the top field of frame N+1.  If I'm right I could play some tricks with what I send to the encoder.  If that works then I go with that and let those who follow figure it out themselves and use my posts as a clue.

John A

Thanks John. Your inference seems to be correct(I got confused when i said that top is ahead as it is captured first).

But this is not codec issue, seems the problem the way buffer is given to the codec.

regards

Yashwant 

Yashwant,

If you can give me one plausible explanation of how I could be sending a captured buffer to the encoder that would cause this I will investigate it.  At this time I am sending the fields from two different captured buffers and the problem appears to be gone.

The thing is that you get no feedback from the encoder as to which field it thinks it's encoding on call to Venc_process.  Now there is something I'm doing that may be involved.  I am using VIDENC1_control to force an IDR frame every 30 frames.  After the call to Venc1_process (twice) I then call the control function to set it back to NA_FRAME.  Perhaps this is causing a bug of sort to occur.

With the now "fixed" encode that looks great on the CRT monitor at work watching traffic out the window, I get a weird anomoly on my 42" LCD screen at home.  I get vertical lines in the video when something occurs like fast movement.  Or even no movement at all, just a bright flash on the video like you might see in a MTV rock video.

John A

OK, here's what I've discovered.  The problem is not in the encoder, it's in the decoder.  After implementing the two frame field swap everything looked great on the decoder.  Then a coworker asked to see it on VLC and the field errors were obvious.

Before I had reported that VLC seemed to be masking the error.  Then Veeranna claimed he saw the problem with my encoded video in both VLC and the Davinci decoder.  So at that point I decided the problem was in the encoder and adopted the field swap solution.  Well, VLC is not masking the error.  And I do not beleive that Veeranna was correct when he said the problem was exhibited in VLC with my file.  If anyone wants to see how the problem looks in VLC I can encode another sample.

So now I'm back to the H264 decoder on the DM368 is broke.  The field ordering is somehow incorrect and I have no idea how to fix it.

John A

Hi John,

I decoded the stream you shared http://dl.dropbox.com/u/3345452/test.264 with DM365 decoder and with JVT decoder both output YUVs are bit exact. VLC is showing trailing edges but less, i think VLC is doing some processing. Try to pause VLC player while playing and observe the video. I also opened this stream in  other analyzers like streameye, VEGA, vprove all analyzers are showing trailing edges.

Veeranna,

You should see trailing edges in the video because it's interlaced.  However I'm seeing an opposite effect between the DM368 decoder and VLC.  When I encode the video normally it looks fine in VLC and moving objects jitter on the DM368.  When I combine field across two frames in the encoder (a kluge) then the video looks fine on the DM368 and jitters in VLC.  VLC and the DM368 are the only two decoders I have to compare at this time.  I can look into d/l and compiling the JVT software.

I've also run my application on the DM365EVM to make sure it wasn't a hardware design issue on our board.  Plus I've decoded the test.264 file with the DVSDK sample app to verify that my app doesn't have a bug in the decode.  I can't encode interlaced with the DVSDK sample encode, so that's the only thing holding me back from proving the decoder is not working correctly.

John A

Veeranna,

I downloaded JVT and decoded the test.264 stream.  I loaded the 420 YUV file into my render app that colors alternating scan line red and green.  The decoded YUV file from JVT is correct.   I.E. the odd lines are advanced past the even lines in horz motion.  However, the images being decoded my the DM368 are not correct on the screen.  The only test left is to use the DM368 to write a YUV file and render it in red/green.

John A

I've decoded the test.264 stream with the DM368 and it does appear identical to the JVT decode.  I guess I'll have to keep scratching my head on this.

John A

Here's one for you...

When I get the display buffer from the decoder I found that it puts the offset x=24 and y=48 in the "dim" structure.  I have a habit of doing weird things when I get stuck with a bug, so I tried changing the y offset to 47.  This also entailed commenting out the asserts in framecopy_accel that check to see that y is even.  Since there is only one line of  chroma for each two luma field lines I expected to see the chroma off by one scan line.  i.e. top field chroma assigned to the bottom field and vice versa.

But what really happened....

The motion is smooth and no jitter on the DM368 decoder.  Looks perfect!!! Except for one thing...

The colors appear (at first) mostly fine.  But I have blobs of wrong colors in various spots.  A vertical bar (<10% of the width) in the center of the screen is gray scale.

What's going on is the chroma is shifted half way across the screen.  The gray bar in the center is the horz blanking.  The color blobs are misc objects with strong colors like sky (blue) or traffic cones (orange) shifted 1/2 way across the screen.

Now I'm at the point where I think the rendering of the interlaced decoded video is flawed.  I've verified the encoder is ok, and now with the JVT and my YUV display app it appears the decoded video is ok.  So now I'm at the rendering step.

So if anyone has any advice on how the 420 video is rendered to the display I am open to suggestions.  The fact that chroma shifted horizontally by half the screen when I shifted Y by one tells me there is more going on in the rendering that I understand.

John A

I see now the basics of getting the decoded video buffer to the display buffer are handled in the Framecopy stuff.  That's where I will focus my effort.  The display code I stole from the decoder sample, so I'll look closer at it as well.

John A

John A,

Did you get anywhere with this problem?  I am having similar issues with a DMAI application using DVSDK 4_02_00_06 and a dm365 board.  I have written several test applications to try and determine where the problem is, but am at a loss right now.   I removed the encode / decode step completely to rule that out.  

I am running with the Resizer in ONE_SHOT mode to capture both fields.  If I do not use the Resizer (Capture UYVY -> Display UYVY) the video output looks great.  Once I add in the resizer  (Capture -> Resizer -> Display) the problems start showing up.  Portions of the video that are not moving still look good, but any objects in motion look jittery or kind of shimmer.  It is quite nauseating to look at.  Almost like you mentioned before, the fields are being output in the wrong order or combined improperly for display. 

I have been debugging this long enough to have a full toolbox of DMAI test apps.  I can dump frames to file, play them back, view them on a PC.  Look at them on several displays.  I just can't figure out what I need to change get rid of the jitter.  And similar to this thread I keep going back and forth in which subsystem to blame (capture, display, resizer).  What really boggles me is that if I dump the frames I capture to a file, and dump the resizer output to a file and play them elsewhere in a YUV viewer they appear identical.  All signs point to a problem in the dm365 display code, but I keep coming back to my use of the Resizer as the trigger.

Using the resizer in continuous mode fixes the problem, but I lose a field, as it line doubles one field rather than combining the two.  I need both fields.

I have also tried banging my head off of several flat surfaces.   Surprisingly,  this did not fix the problem either.

I want what your original post described.  NTSC composite capture -> encode -> RTP -> decode -> NTSC composite display.  I am frustrated with DMAI.  Did you find a better way? Any help would be greatly appreciated.

Joe