The documentation states that the (INDTXSCWB) scaler is reconfigurable while in-use. (In the OpenMAX user guide, under "Features supported" it states "Support dynamic resolution change on both input and output side".)My question, then, is how to actually access this dynamic resolution change feature? Is there a demo program using it?Calling OMX_SetConfig on the handle with index OMX_TI_IndexConfigVidChResolution and a structure of type OMX_CONFIG_VIDCHANNEL_RESOLUTION looks like an interface which would work, but doesn't seem to in practice. Whenever I try to use it after the component has been running for a while, the SetConfig succeeds but after that the scaler stops returning buffers entirely (neither input or output buffers are returned, and the pipeline feeding it just locks up). The VPSS logs (from loggerSMDump) do not say anything about the change, containing only the initial setup of the scaler. The various demo programs (decode_display, etc.) do use this function, but only for static configuration before actually passing anything to the component.- Mark
See attached for a patch to ilclient.c in the decode_display example showing the suggested method not working. This hacks it to call OMX_SetConfig OMX_TI_IndexConfigVidChResolution with a modified resolution after 100 frames of scaler input. In my testing, it works fine up to that point but the scaler always hangs after the call - it accepts more buffers, but never returns any.
- Mark
3581.decode_display_dynamic_config.patch.txt
Hi Mark,
It is not clear that, you want to change input stream dynamically or want to change output scaling ? for scaling o/p port properties needs to be changed. If you want to take a portion of picture, you can use crop width/height for doing so.
Regards
Vimal
Hi Vimal; thank you for replying!
Vimal Jain It is not clear that, you want to change input stream dynamically or want to change output scaling ? for scaling o/p port properties needs to be changed. If you want to take a portion of picture, you can use crop width/height for doing so.
Ideally I would like to reconfigure both input and output, but in this case input resolution is the interesting one. Cropping the image is not the intent at all, though I admit it ends up doing that in the example I gave (the example was intended as a simple demonstration that trying to configure it like this just hangs the scaler, rather than that it's exactly what I want to do).
Consider a scaler which is scaling decoded video up to be sent to an output. The H.264 stream being fed to the decoder may change resolution at any moment, so the scaler needs to be able to change its configured input resolution when that happens in order for the output to be correct.
For example, the scaler input and decoder might be configured with a maximum resolution of 1280x720, and the scaler output at 1920x1080. Initially the H.264 stream going in to the decoder is 1280x720 so the scaler input is configured to 1280x720 with pitch 1408. Suppose now the stream changes to be 512x288, then when the first frame of the new resolution comes out of the decoder the scaler needs to be reconfigured to have input 512x288 with pitch 640 but otherwise continue exactly as before (same buffers, same output).
How might this be achieved?
Thanks,
Mark,
in OMX height/width/pitch is port format, and unfortunately can not be changed in execute state. Scalar was provided these as runtime parameter, to take from buffer header. However standard OMX header does not alllow these fields in buffer header ! so Scalar ends up taking those values from port properties. Typically if buffer sizes are same for whole stream, decoder should be able to decode smaller resolution with buffer-pitch, and then crop width/height alongwith startX, startY can be used to cut actual video from the buffer and then scale it up.
Vimal Jain Typically if buffer sizes are same for whole stream, decoder should be able to decode smaller resolution with buffer-pitch, and then crop width/height alongwith startX, startY can be used to cut actual video from the buffer and then scale it up.
Typically if buffer sizes are same for whole stream, decoder should be able to decode smaller resolution with buffer-pitch, and then crop width/height alongwith startX, startY can be used to cut actual video from the buffer and then scale it up.
Ok, that sounds like a good approach. See attached for another attempt (as above, it's a patch to ilclient.c in the decode_display example; 720p only still). Now I'm setting the StartX, StartY, CropWidth and CropHeight while keeping the Width, Height and Pitch fields the same as they were at startup. This does not hang the scaler, but it also doesn't appear to actually do anything - the output video stays exactly the same. Am I missing some other parameters I need to set somewhere?
0333.decode_display_dynamic_config.patch.txt
sorry, cropping/scaling is broken in last release, and it takes only port properties in effect. Issue is identified, and we will be able to provide fix in next release.
Hi Vimal,
Thank you for that answer. When will the next release, with this feature working, be available?
As a strongly related question, will this dynamic reconfiguration also work on the noise filter component? I am unable to test anything about the noise filter because it doesn't work at all in 5.04.
This would be useful for the capture -> scale -> noise filter -> encoder use case, to allow different encode resolutions from the same input source (using the noise filter in bypass mode for its 422 -> 420 conversion, since the scaler can only output 422 and the encoder can only input 420).
NF does not have run time change parameters. You could use DEI path for capture and scaling, It can provide 420 output for encoder.
Is DEI dynamically reconfigurable as above?
Also, what are the resource use implications of using it? In an application already using using three instances of the scaler, will it continue to work? (Since it generates two outputs (one of which will just be discarded in this case), I am guessing that it must use up two scaler instances.)
I have now tried scaling with DEIH and it mostly works.
Some observations:
- DEIH seems to coexist perfectly well with existing INDTXSCWB scalers.
- Dynamic reconfiguration doesn't appear to work here either (I assume this is the same problem as with the vanilla scalers).
- When downscaling to an output with pitch not equal to width, the two output (luma and chroma) planes overlap, which messes up large sections of the output image. It looks like it chooses the start address of the chroma plane based on the width and height inputs initially (ignoring the pitch), so this would probably be solved by working reconfiguration (by setting an initial maximum size with pitch equal to width and then switching to the actual intended size immediately after).
- The fact that it is pointlessly generating an additional 422 output which is discarded doesn't appears to harm anything, though to avoid wasting memory bandwidth it would be nice to switch this off.
So, comment on dynamic reconfiguration of DEIH too?
Hi,
DEI has dynamic reconfiguration provision, but probably is not verified as you observed. Please note De-interlace algorithm has to be in bypass mode for dynamic reconfiguration. (bAlgbypass = 1).
for downscaling, do you want to retain original pitch and scale the buffer ? This is with SC or DEI ?
DEI component is using dual-out driver, however you can turn off one port to disable the dual o/p.
Thank you for your reply.
Vimal Jain for downscaling, do you want to retain original pitch and scale the buffer ? This is with SC or DEI ?
This is with DEI. The aim is to end up with whatever pitch combination the encoder accepts. I have tried setting videnc2DynamicParams.captureWidth on the encoder, but it seems to always use the pitch from the original port definition anyway (though it does respect the inputWidth and inputHeight parameters). Therefore, I am trying to make the DEI output use the encoder input pitch. Setting Frm1Pitch on the output of the DEI component has the right result for luma (in that the encoded video has correct luma), but the chroma is not right - it seems like the two planes are overlapping somehow.
Note that this problem may instead be on the encoder end, if it is finding the location of the chroma plane incorrectly when inputWidth and inputHeight are changed. (In all cases the luma output is correct, so it is definitely mostly working.)
I will have a more careful look at the buffers coming out of the DEI component to try to work out where the problem is, but if you are aware of any issues around this it would be very helpful.
Vimal Jain DEI component is using dual-out driver, however you can turn off one port to disable the dual o/p.
Thank you for that information, I will make this change to avoid writing the 422 output at all.
I have a further question regarding scaler use. It seems that the VPSS firmware only supports four scalers of either kind (DEI or INDTXSCWB), and crashes when a fifth is allocated. However, I believe there should be more than this available (the OpenMAX manual suggests that there are five, labelling them SC1 to SC5). Are more scaling resources available? If so, how can they be accessed?
I have resolved the incorrectly placed chroma plane issue.
When DEI is configured (via VidChResolution) to have a output width and height which is different from the original port width and height, it still calculates the position of the output chroma plane based on the original values rather than the configured values.
For example, if set up with 1280x720 (1280 pitch) ports and then configured to 768x432 (1280 pitch), the luma plane starts at offset 0 as expected but the chroma plane starts at 1280*720 = 921600 rather than 1280*432 = 552960. This is a made worse by the fact that on the output buffer the nFilledLen is totally wrong, as it shows 768*432*3/2 = 497664 in this case. So, when fed into the encoder, it calculates that the chroma plane should be at 2/3*497664 = 331776, and ends starting halfway through the luma plane. To work around the problem, the filled length of the DEI output buffer should be ignored and instead the expected value from the original port setup should be used (that is, 1280*720*3/2 = 1382400). With this change, the encoder will then look for the chroma plane at 1382400*2/3 = 921600, which is correct.
Following further testing around this, I think that my original suggestion that DEI dynamic reconfiguration does not work was in fact wrong (somewhat confused by the above issue). This does now work for at least simple cases - I will investigate more complex cases and repeated reconfiguration in the next few days.
Thanks..
that points out to issue in filledLength calculation in DEI component. We would look into that issue.