DM365 hangs up when video stream has missing frames

Himanshu,

Codec standlaone test with missing frames had passed for us during the test. Can you send a sample stream which makes the codecs hang, we would like to check out the problem. DM365 codec is designed to work in error network.  Also, please send the exact vesion number of the codec used.

regards

Yashwant

I'm having essentially the same problem and was going to post it before finding this problem report.  Unfortunately, I don't have a sample stream to provide you because of the way our system works.

In our system we've implemented a transport stream playback utility.  The transport stream contains h.264 video, aac audio (using ittiam codecs), subtitles, and a fourth user-defined PID which has periodic packets containing our system time.  The audio and video were encoded and decoded on the same dm365.  I'm using dvsdk 2.10.01.18, with dm365_codecs_01_00_06, which has h264 encoder version 01.20.00.05 and h264 decoder version 01.10.00.04.  We're currently not attempting to display subtitles with the decoded video.

Our playback utility provides a "skip" feature, which allows the user to use the UI to skip forward or backward in the stream, and the skip value I was testing with was 20 seconds.  So under normal operation we read the stream file and decode it in 192K chunks.  When the user presses "skip", we search through the file looking for our own timestamps to determine when we've jumped forward or backward the correct amount of time, then resume decoding from that new point in the file (which was 20 seconds forward or backward from the last-decoded data).

I found in testing that I was seeing a variety of problems on the dm365.  Eight or nine times out of 10 I could skip with no problems, but at other times the dm365 would hang, with no console output.  Other times I would get a kernel "oops" which was sometimes an alignment trap and other times was due to dereferencing a null pointer (which according to the stack dump was always in an ISR).  The most common problem was the hang.  Skipping forward and skipping backward seemed to produce the crash with equal frequency.

I was able to eliminate the crashes by doing two things.  The first was to make sure that I always decoded the remainder of the current PES frame (audio or video) before performing the skip.  Prior to this change, it was likely that we decoded only a partial video frame before jumping to the new point in the file (and the new point was guaranteed to *not* be in the middle of a video or audio frame).  After implementing this change, I still saw occasional crashes, so I'm not sure how much (if any) I benefitted from this change.

The second fix was to make sure that after skipping to the new point in the file, I started video decode with an IDR frame.  We're encoding video with IDR frames occurring once per second, so after skipping to the new point in the file during decode, I would then search forward for an h.264 IDR frame before resuming decode.  This eliminated the problem, which seems to indicate that this was a problem with the video decoder and not the audio decoder.  Of course, without ever understanding the true cause of the crashes, I can't know that I've really fixed it.  It's circumstantial at this point.  But the problem went from being easily reproduced to my no longer seeing even one crash.  So the cause of the crashes seems to be due to decoding non-IDR video frames that have absolutely no relation to previously decoded data.

Based on what little I know about transport streams and h.264, I thought that even my crude initial implementation was valid and shouldn't cause kernel crashes.  Further, even with my "fix" in place we seem to be vulnerable to legitimate dropouts of data.  Unfortunately, due to the nature of my random skips through the data while testing, I can't reproduce specific crashes, I can only create an environment that crashes fairly often.

Any feedback would be appreciated.

Regards,

Eric

Hi Eric,

Thanks for the detailed analysis. The decoder should not crash with streams having frame loss. Ideally, one does not need to sync the decoder at the next IDR extrenally. Though we have tested this scenario, we will try to add more test cases to recreate the issue reported by you.

In the meantime, I also reccomend you to try out the new ver 2.00 codec which can be downloaded from http://software-dl.ti.com/dsps/dsps_public_sw/codecs/DM365/index_FDS.html. In case you are using base mode, the library should work fine without application changes. In case you are using extended parameter of decoder, you may have to do few lines of modification to take care on the new parameters.

regards

Yashwant

Hi,

Unfortunately, we are not able to reproduce the issue. We generated fresh error streams by removing frames from different types of 264 streams. All these streams decoded without any hang in our system.

So now we totally depend on you to provide a stream through which can reproduce this behaviour, I understand it may be difficult since your app may not have file i/o, but if you can get it with some minor tweaks in your app, it will be great. Otherwise we find it difficult to proceed on this issue.

regards

Yashwant

One more point to note is the way buffer management is done by the application. Since the problem goes away with IDR frame, it seems it is linked to that. H.264 codec can lock buffers for future referencing(This is as per H.264 spec), if you do seek to a non IDR frame, the codec may still have some old locked buffer. It may result a scenario when the system runs out of buffer and produces a hang or crash(codec getting incorrect i/o buffers etc). Did  you check if this is not the case ? WIth IDR frame, codec releases the old locked buffers and hence the system may not be seeing this crash.

regards

Yashwant

Yashwant,

can you send me your email address, I can send you the file.

Also, please note:

"Sorry we didn't make my issue clear.  Missing frames is not the problem.  It's missing data in the frame that causes the crash.  So if you  can randomly delete data in a frame I believe you will see the h.264 codec crash."

regards

Himanshu,

Have you tried attaching the stream in the post ? You can do it by going to the "options" tab.

regards

Yashwant

Yashwant:

I will send the files to you via email.

Thanks,

Pradhyum Ramkumar

Himanshu,

We downloaded the stream sent by Pradhyum and analyzed it.

 In decoder standalone test app: The stream decodes the full ~3600 frames without any hang or crash, doing error concealment where-ever needed.

In DVSDK 3.1 demo: The stream plays for 1mnt 20sec (doing error concealment where-ever needed) and then stops giving out o/p.

 We suspect that the problem in the demo app and it may be because of incorrect buffer handling by DMAI. Looks like DMAI is not able to reuse the buffers which are released immediately by codec. 

 By any chance, is your application based on DMAI ? if yes, we can suggest a workaround which you can try.

regards

Yashwant

Yashwant:

I like to bring up this topic again. We met the exact same problem. Our code hang in decoding stage if some packets are missing.

Could you explain the workaround you suggested?

Thanks very much

Regards

Jun

Attached is the workaround in DMAI which should fix it.

--------------

In Vdec2.c – Added a new function “Vdec2_process_new”. This is very similar to “Vdec2_process” except that it has one more argument outArgs. (Please change the name of the function to something else)

In Vdec2.h – Ln no 151. Added function declaration of “Vdec2_process_sithara”

In decode/video.c – The changes I added are under “#ifdef CorruptFrm” at ln 157 & 424

You can find Vdec2.c and Vdec2.h at dvsdk_2_10_01_18/dmai_1_21_00_10/packages/ti/sdo/dmai/ce

 --------------

Also, Please make sure that you are on the latest version of decoder - ver 2.00.00.11( http://software-dl.ti.com/dsps/dsps_public_sw/codecs/DM365/index_FDS.html )

Yashwant:

Thank you very much! problem solved. Now the decoder can recover from any missing packets. It really helps.

thanks again.

Jun