Problems in handling EventPortSettingsChange Event on OMAP 4430

Hi Pradeep,

Based on the codec error in the logs attached, it seems like there is a problem with PPS parsing as mentioned below. It seems like even the OMX_IndexConfigCommonOutputCrop is not supported. Can you try without cropping by decoding it at full resolution and see if it make any difference?

ERROR CODE : 10000000000000000001000000000101

0 - No error-free slice header detected in the frame
2 - Error during PPS parsing
12 - Header problem/corruption
31 - Indicates that a gap is detected in frame_num for a stream with gaps_in_frame_num_value_allowed_flag 1 in SPS

Also, ensure if the stream has no errors.

Thanks & Best Regards,

Venkat

Hi Venkat,

 I disabled configuring the cropping settings but still i'm seeing the same kind of errors. As per my observation, the Decoder component is raising the EventPortSettingsChangeEvent for the second frame also (i,,e P frame ) which is submitted by me consider the following ducati log... This is a fresh log different from the previous log although the error is same 0x1421

[    498.556] [MNU=528] rm_plugin_res/src/rm_pluginpower_resource.c:[1127]:[L3BUS]: Succesfull request of BW constraints
[    498.556] [MNU=529] omx_videodec_common/src/omx_video_decoder.c:[994]: 1.Codec Create for ivahd_h264dec
[    499.396] [ERR=530] omx_videodec_common/src/omx_video_decoder.c:[1872]:width = 176. expected width = 256, height =  144, expected height = 256
[    499.397] [ERR=531] omx_videodec_common/src/omx_video_decoder.c:[1889]:Dynamic port reconfiguration triggered since small o/p buffer size provided in first frame decode
[    499.441] [MNU=532] rm_plugin_res/src/rm_pluginres_ivahd.c:[393]:UpdateResourceRequest:: User requested IVAHD Mhz:10
[    499.441] [MNU=533] rm_plugin_res/src/rm_pluginres_ivahd.c:[394]:UpdateResourceRequest:: User requested IVAHD Latency:400
[    499.441] [MNU=534] rm_plugin_res/src/rm_pluginres_ivahd.c:[402]:UpdateResourceRequest:: Setting IVAHD Mhz:10
[    499.442] [MNU=535] rm_plugin_res/src/rm_pluginpower_resource.c:[1156]:[IVAHD]: Succesfull request of Mhz constraints
[    499.443] [MNU=536] rm_plugin_res/src/rm_pluginres_ivahd.c:[406]:UpdateResourceRequest:: Setting IVAHD Latency:400
[    499.444] [MNU=537] rm_plugin_res/src/rm_pluginpower_resource.c:[1169]:[IVAHD]: Succesfull request of latency constraints
[    499.444] [MNU=538] rm_plugin_res/src/rm_pluginres_ipu.c:[316]:UpdateResourceRequest:: User requested IPU[1] Mhz:40
[    499.444] [MNU=539] rm_plugin_res/src/rm_pluginres_ipu.c:[317]:UpdateResourceRequest:: User requested IPU[1] Latency:400
[    499.445] [MNU=540] rm_common/src/rm_constraint.c:[118]:RM CONST:: Warning: Reaching Max constraints
[    499.445] [MNU=541] rm_plugin_res/src/rm_pluginres_ipu.c:[325]:UpdateResourceRequest:: Setting IPU[1] Mhz:200
[    499.445] [MNU=542] rm_plugin_res/src/rm_pluginres_ipu.c:[329]:UpdateResourceRequest:: Setting IPU[1] Latency:10
[    499.445] [MNU=543] rm_plugin_res/src/rm_pluginpower_resource.c:[1096]:[IPU]: Succesfull request of latency constraints
[    499.446] [MNU=544] rm_plugin_res/src/rm_pluginres_l3bus.c:[262]:UpdateResourceRequest:: User requested L3BUS BW:9 MB/s
[    499.446] [MNU=545] rm_plugin_res/src/rm_pluginres_l3bus.c:[267]:UpdateResourceRequest:: Setting L3BUS BW:1009
[    499.447] [MNU=546] rm_plugin_res/src/rm_pluginpower_resource.c:[1108]:[IPU]: Succesfull request of BW Constraints
[    499.447] [MNU=547] rm_plugin_res/src/rm_pluginpower_resource.c:[1127]:[L3BUS]: Succesfull request of BW constraints
[    499.492] [ERR=548] omx_videodec_common/src/omx_video_decoder.c:[2257]:
[    499.492]  Process function returned an Error...  
[    499.492]
[    499.492] [ERR=549] omx_videodec_common/src/omx_video_decoder.c:[2259]:Codec Extended - 0x1421
[    499.492] [ERR=550] omx_videodec_common/src/omx_video_decoder.c:[2262]:Input Buffer Size provided to codec is : 14235
[    499.492] [ERR=551] omx_videodec_common/src/omx_video_decoder.c:[2264]:Frame count is : 1
[    499.493] [ERR=552] omx_videodec_common/src/omx_video_decoder.c:[2266]:Bytes consumed - 14235
[    499.493] [MNU=553] omx_videodec_common/src/omx_video_decoder.c:[2269]: Codec Used is ivahd_h264dec
[    499.493] [ERR=554] omx_videodec_common/src/omx_video_decoder_utils.c:[1436]:Dynamic port reconfiguration triggered

The first frame is an i-frame of 1316 for which the EventPortSettingsChangeEvent is generated. My first question why is the omx component not giving the correct o/p w and h detected ? In my case it is a CIF image (352 , 288) where as the o/p which the ducati log shows is 256 and 256 ?

After receiving the event, i transitioned the component to Idle state, then EmptyBufferDone () is called with filled length as 1316. FillBufferDone () callback is also called but the value of FilledLength is 0. why is that happening ? As per the omx standard, the decoder should fill the number of bytes  decoded by the decoder ?

The next frame which is given to decoder is 14235 bytes (it is a P frame) as seen in the ethereal trace.

The decoder did not decode this frame and shows the error 0x1421. Surely it does not contain the SPS/PPS parameters because it is a P frame .

Now why decoder is triggering the EvenPortSettingsChangeEvent ? It should give an error and return back ? Why is it waiting for the App to configure the port once again ?

The ducati log shows that frame count: 1 but according to me it should be 2 because the I-frame is already decoded ( i,,e 1316 bytes) and the omx layer has generated PortSettingsChangeEvent for decoding that frame.

Does it mean that OMX Decoder starts once again to search for SPS/PPS parameters once it has generated EventPortSettingsChangeEvent ? Does it mean App should provide SPS/PPS once again after EventPortSettingsChangeEvent is generated ?

regards

pradeep

Could you check next steps and see if error is gone?

1. Load OMX component and set it to OMX_StateExecuting and send output buffers for processing.

2. Send first input buffer containing only SPS+PPS.

3. Received port reconfiguration event OMX_PortSettingsChangedEvent.

4. Disable respective port using returned index, it must be output port.

5. De-allocate output buffers in order to get port disabled.

6. Read port definition values and update component according.

7. Enable the disabled port allocating respective buffers to get command complete.

8. You could receive a second event indication cropping values do the same.

9. Component must accept next input buffer containing first Iframe, no SPS+PPS needs to be send.

10. Continue processing input and output buffers.

Hll  Manuel,

             I've checked and the error still persists. Please find my comments inline with your suggestions

Manuel:

1. Load OMX component and set it to OMX_StateExecuting and send output buffers for processing.

2. Send first input buffer containing only SPS+PPS.

3. Received port reconfiguration event OMX_PortSettingsChangedEvent.

4. Disable respective port using returned index, it must be output port.

5. De-allocate output buffers in order to get port disabled.

pradeep

All the above 5 steps are successful. empty buffer done and fill buffer done call back functions were called by omx layer after executing the step  4.

Manuel:

6. Read port definition values and update component according.

pradeep:

I retrieved the index port parameter for out put port. The values provided by the OMX library is  nFrameWidth =256, nFrameHeight = 256 same as that shown in the ducati log . I think the nFrameWidth,nFrameHeight should have been updated with the actual image width and height detected by the decoder. In my case it should be 352 and 288. I tried with VGA resolution in another call but the behavior is the same i,,e the values of w and h which are received is nFrameWidth = 256 and height = 256 .

                     After 6th step i should have received the correct image sizes for w and h but i'm not seeing the correct values, This step is not OK as i was expecting the correct image w and h and buff sizes but still i went ahead with the steps to check the behavior

Manuel:

7. Enable the disabled port allocating respective buffers to get command complete.

pradeep:

Step 7 was successful. o/p Port enabled and allocated  buffers for the o/p port. sent command to transition to executing state and component was in executing state

Manuel:

8. You could receive a second event indication cropping values do the same.

pradeep:

Received cropping indication. Followed the steps 1-7 once again. steps 1-7 were successful. The values of w and h returned by the omx component when called with IndexPortParamete for the o/p port are 256, 256 instead of 352 and 288.

Observation: sometimes the width and height returned by the GetConfig command called with OMX_IndexConfigCommonOutputCrop index was 352 and 288 and most of the times it was 176 and 144 which could be the default values. 

Does it mean the actual width and height detected by the decoder is provided when  GetConfig command called with OMX_IndexConfigCommonOutputCrop ?

Manuel:

9. Component must accept next input buffer containing first Iframe, no SPS+PPS needs to be send.

pradeep:

 The component still sends EventPortSettingsChange Event again after executing step 9. My App blocks till i follow the same steps from 1 to 8. It goes in a loop

manuel:

10. Continue processing input and output buffers.

pradeep:

Does not reach to step 10. problem at step 9 and goes in a loop from step 1 to 9

Is there any demo or an example program which handles the above steps mentioned by you and succeeds ? I went through the VTC application provided in the link 

http://review.omapzoom.org/#/c/19748/16/test/VTC/IOMXDecoder.cpp

It just frees the o/p buffers and configures crop values i,,e with respect to my app, it has code till step 7 and not from step 8

Pradeep;

Just to confirm, are you using 4AI.1.4 or later release? and can you share the file for analysis?

Manuel;

Yes i'm using 4AI.1.4 release. The ducati trace shows as 4.AI.4 I've attached PortSettingChange file which contains a function that handles port settings change/ crop events. I've added comments at appropriate places. you can grep for "step" to map the above steps mentioned by you and the actual code. I've taken a known file which has 1 h264 i-frame. The file is dumped in bin mode but i changed the extension to .txt format in order to attach the file. you can do a hexdump to check the contents

please let me know if you find any problem

0243.PortSetting_function.txt

2161.decoder_input.txt

pradeep

Pradeep,

[Reply to your first post]

0x80001005 Corresponds to omx error "OMX_ErrorBadParameter"

Looks like you have tried to call getConfig with the index OMX_IndexConfigCommonOutputCrop on input port while it is supported only on output port.

Shivraj,

                i've corrected to configure the crop parameters for  out port. Configuration was successful but still i'm unable to get o/p from decoder.

regards

pradeep

Pradeep;

After analyzing the stream the only difference I found was the flag nFrameCroppingFlag, it is zero that means no cropping values are stored in this file.

By checking the issue it could be that at some point this flag is not correctly used and that results in frame limits to be passed and incorrect values are used for calculus. That this can result in incorrect width and height values.

I am trying to check this with proper team it could take some time but mean time could you try using another file or files in your tests? and from where are you capturing this stream or some special configuration is used if may I ask? or if it is possible to share in the forum, usually files comes with some value here to make H264 format match a multiple of 16 resolution or with zero values with the flag set.

next is a parsing values that I got

nProfileIdc 66 0x42
nConstraintSet0Flag 1
nConstraintSet1Flag 0
nConstraintSet2Flag 0
nReservedZero5bits 0
nLevelIdc 13
nSeqParameterSetId 1
nLog2MaxFrameNumMinus4 10
nPicOrderCntType 2
nNumRefFrames 1
nGapsInFrameNumValueAllowedFlag 0
(*nWidth) 352
(*nHeight) 288
nFrameMbsOnlyFlag 1
nDirect8x8InferenceFlag 0
nFrameCroppingFlag 0

By checking shared code I have next observations:

1. My mistake, when doing port reconfiguration the next buffer that is send must be SPS+PPS just after reconfiguring this to let Ducati code to re-read the values and compare them, when everything is ok then event is not returned and then Iframe needs to be send. In OMX specification they mention this procedure and in general as after sending the first buffer it is required to wait for it to comeback from codec before sending the next buffer, if while this time you receive a port reconfiguration event then it needs to be attended and continue with the process. I repeat my mistake.

2. In code you are calling next state OMX_StateIdle and then disabling and enabling the ports, then moving to OMX_StateExecuting again, can you try not moving to Idle state and by consequence not calling Executing state neither? there are 2 procedures to handle port reconfiguration,

a) Disabling-Enabling ports with respective de-allocation and allocation of buffers, and in the middle reading modified values.

b) Moving the component in next sequence being in OMX_StateExecuting->OMX_StateIdle(freebuffers)->OMX_StateLoaded-(read values)-OMX_StateLoaded->OMX_StateIdle(allocatebuffers)->OMX_StateExecuting.

I never tried moving to OMX_StateIdle state and then to OMX_StateExecuting, not sure of the implications or changes that could affect.

And add the change that Shivraj mentioned.

By #1 it could be that second row read values are incorrect since it is an Iframe and not related to mentioned flag.

Hi Manuel,

                    finally  I could get the output from decoder. I was comparing my code with the demo code given in VTCTestLoopback (4.Ai. 7) . I noticed  2 differences.

1. The first i-frame which i had given to decoder contained sps + pps (23 bytes )+ I-frame (1293 bytes )=1316 bytes. I used to call the omx_empty_buffer with 1316 bytes where as test app used to give only sps + pps (23 bytes). So what would happened in my case is that the Decoder would parse the complete 1316 bytes instead of stopping at 23 bytes (i,,e end of sps + pps) and used to thrown an error 0x1421. i,,e sps and pps not found.

                    So when i had given complete i-frame with or with out sps + pps after EventPortSettingsChangeEvent is trigged, it used to trigger the EventPortSettingsChangeEvent once again since it actually did not decode the frame. this results in event being triggered continuously

                    When i changed the nFilledLen to 23 , the decoded started decoding the sps + pps. i,,e when i called omx_empty_buffer with 23 bytes (sps + pps) instead of 1316 bytes ( sps +pps + i-frame).

2.    The TestApp was configuring the w and h of the frame to be decoded for i/p port. The w and h of the frame contained in the stream provided to the TestApp is same as the width and height configured in the input port. Where as in my video phone Application, I was not configuring the width and height of the frame to be decoded thinking that the decoder should identify based on the stream given to it (i,,e it should automatically determine the w and h from the sps and pps ) since the stream can change across different calls. What i mean to say is that in case of a live video call, the remote party can send the h264 stream in either a qvif or cif or vga or qvga format and so i can't determine the the w and h of the frame before hand.

       pCompInfo->sOmxInPortDef.format.video.nFrameWidth = CIF_IMAGE_WIDTH;
       pCompInfo->sOmxInPortDef.format.video.nFrameHeight = CIF_IMAGE_HEIGHT;

 when i configured frame width and height of the input port to cif and i forced the remote end to send  CIF stream,  and configured as step 1, the decoder started working and i get o/p from decoder. To re-check if  frame w and h config is essential for an i/p port, i changed the frame w and h of i/p port to VGA_width and VGA_height, gave cif i/p stream. After his change, the port re-configuration is triggered continuously, If i force the remote end to send VGA stream without any change in my video phone app, the decoder works perfectly.

              this means that the frame w and h fo i/p port should be configured and we need to give the stream which matches the i/p port configuration. why is it so ?

The o/p of decoder contains the padded bytes similar to that of DM 8148. Is there any documentation which states the way in which decoder o/ps  w and h, the padding mechanism of decoder. Also is there any test app or an OMX component which takes the decoder o/p,converts to rg 565 and displays it on the LCD ?

Thanks & regards

pradeep

Hi Pradeep,

On OMAP4460, the DSS is capable to converting the YUV inputs to RGB by itself if you use one of the Video Overlays to display on LCD.

In case you want to do explicit conversion using the A9 by accessing the buffers, you can refer to the following posts in regards to accessing the YUV content and accordingly implement the conversion using YUV to RGB color conversion matrix.

http://e2e.ti.com/support/omap/f/849/p/193199/695299.aspx#695299

http://e2e.ti.com/support/omap/f/849/p/189036/678397.aspx#678397

Coming to input port configuration, as per my understanding as long as you chose to configure the input port with lower resolution, then when the remote end starts sending input which is either the same or higher resolution, you would receive port reconfiguration event to reallocate the buffers accordingly which would have padded size as per the codec feedback, then it should not be a problem.

In the above test you had, I think port reconfiguration did not take place since you had already allocated for VGA which is way above the padded CIF requirements.

Thanks & Best Regards,

Venkat

Hi Venkat,

          After going through the posts mentioned by you, I could find how i can convert the packed yuv 420 semi planar to yuv420 planar. the yuv resizer function is used only to resize the yuv frame but not convert to RGB. Both of these two are done in s/w. I think the Display sub sytem has 3 overlays of which 1 can accept nv12 buffer  to display on LCD or write back to memory. I'm thinking the video overlay accepts nv12 converts to rgb 565 and then displays on LCD. so instead of displaying on LCD directly can it write back to memory.

In short i want an h/w accelerated color converted  which can convert nv12 to rgb 565 and output to memory instead of directly displaying on LCD. Is it possible ? If so please let me know how to do it  else confirm that yuv --> rgb conversion should be done in s/w ?

regards

pradeep

Hi Pradeep,

I was giving those references primarily for you to understand how you can access the buffers. You are right, DSS on OMAP4 has 1 Graphics, 3 Video Pipelines and 1 Writeback pipeline.

Yes, you can use the write back pipeline to read the NV12 input from the pipelines and output the RGB 565 on to the memory.

You can refer to the OMAP 4460 TRM, the following section 10.2.4.11 on "Writeback Pipeline" to understand the register attribute settings required for capturing the RGB565 output to memory.

Please see the DISPC_WB_ATTRIBUTES register has a FORMAT field which defines the pixel format that need to be written onto memory.

I hope this clarifies your queries.

Thanks & Best Regards,

Venkat

Hi Venkat,

           Thanks for response. I want to know if there is any omx component  which takes an i/p buffer in some format like nv12 and displays on LCD or if there is any demo app which can show how to do color conversion. If else could you tell me which overlay can be used to display on LCD and how to program my app to use it ?

regards

pradeep

Hi Pradeep,

If you have downloaded the latest 4AI.1.6 or later android releases, you will find one VTC Loopback Test application at the following relative location hardware/ti/omap4xxx/test/VTC which takes input from camera and runs the VTC loopback.

You can modify that application to read it from a file as well and use it.

Alternatively, there are some dsscomp tests which can be used to read NV12 inputs with scaling and rotation support to display. I need to find the exact information regarding these with R & D team and then update you accordingly later.

Thanks & Best Regards,

Venkat

Pradeep;

Was your initial issue resolved? or correct answer provided? can you verify corresponding post? please.