This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

4:2:0->4:2:2 H.264 artifacts

So I have some more information on the issue that we are seeing at a customer.  What they are trying to do is reuse their  IP camera hardware if they can as a decode unit in another part of the store; specifically at a security guards desk.  So ideally they’d like a DM6446 footprint-compatible solution; and in truth; this may come down to SW anyhow.  What they are doing is trying to decode a stream from the IP net cam; they have DMVA2 and a DM6446 version; both have the same issue.  

The h.264 decoder they use is whatever it is we ship with our DVSDK v1.20 for the DM6446.  Now, the issue is; when they decode the stream, and blow it up to a 42” monitor; it looks pretty bad; like the photo shown below.

Now we are not 100% sure why it looks so bad; but the jagged edges on the diagonal lines suggest one of two things to me- either there is an outright incompatibility in the net cam output and the DM6446’s ability to decode it; or there is an artifact being introduced at the DM6446 decoder end.  Normally I would have described this as an interlacing artifact, but if you look close at it it almost looks like the diagonals got the wrong way within the artifact range, suggesting to me something more sinister like an endian error in the 4:2:0 to 4:2:2 conversion process on the backend.  Here’s why I get to that conclusion.  In the course of debugging this; we have found that:

 

The imager itself outputs 4:2:2 video; and if we connect that direct to the TV it looks great (no encoding, no decoding).  Now we know that the DM6446 ENCODER software (the camera side here) only supports 4:2:0 input, so we use the resizer to convert the 4:2:2 to 4:2:0 before the RTSP stream is created.  Now of course this was suspect; but, if we ‘tune’ the that stream with a PC video decoder it still looks fine; and as such, we are pretty confident that the encoded 4:2:0 stream is good.  The problem is, that we really need to get back to 4:2:2 to get to the TV; so , if we replace the PC-based decoder with another DM6446 and try to decode with that; and set up for a 4:2:2 output, we get this squirreled up result.  What I don’t know about our SW decoder is if this 4:2:0 (which it would natively support) to 4:2:2 conversion is being done in SW or is also trying to use the resizer (I’ve always understood the resizer to be on the video input side, so I suspect this is more a brute force SW implementation on the decoder end); but in any case there are clearly artifacts that are not present when using a PC-based decoder.  

 

Just a a sidebar; we have tested the MJPEG encoder and decoder because those DO support 4:2:2; and it looks great; but the problem is they need to use h.264, and hence our issue.

 

  • Well that almost worked... let me try to paste that picture in here again... crossing my fingers

  • So adding to my own post, on behalf of Brad Griffis, who also had the following ideas on this issue.

    Brad wrote:

    Dusting off the old signal processing cobwebs, I recall that generally speaking when upsampling/interpolation you perform zero insertion followed by a low-pass filter.  I’m guessing a bit, though I expect there’s something similar when performing chroma upsampling.  Looking at the image from Dave it looks like there is “horizontal smearing” of all the diagonal lines.  I also see in the adjacent line that there is a corresponding area of white:

    Notice that the white that sits on top of the “smeared black” is NOT the color of the door .  That looks to me like the filtering operation is not happening properly, i.e. we see extra stuff being added but then not filtered/mixed appropriately.

  • To me it looks like your algorithm is working on a box of pixels and applying a mirror effect (not sure if it's horizontal or vertical) which is causing problems with diagonal edges. You'll note that there are no artifacts in the perfectly horizontal/vertical edges so I reckon this is what's going on. I'm not convinced it's an endian error as there seem to be more than a few bytes that are reversed in each "box".
    Ralph

  • Ralph,

    Thanks- soooooo....... this isn't my algorithm...this is what ships with the DVSDK.  I don't know who specifically developed the 4:2:0->4:2:2 conversion routine (tho is me admittedly now ping this on that SW component, but I'm pretty sure that's where the issue is); but I did engage a 3rd party friend of mine who has a SW decoder with 4:2:0->4:2:2 converter running on AM335x and that does not show the issue' the decode stream looks great, just farm rate limited due to the lack of real decoder HW that the DM365 has.  This issue seems to squarely lie in the depths of the DVSDK algorithms TI ships.  

  • Oh I see. Well if you don't get a sensible answer from TI, I'd suggest feeding the video through a codec to reverse the order of the pixels.