gst-openmax and openmax on ics and omap 4

Aaron Pope

Hi, everyone!

I want to use the gst-openmax and omx_h264enc to encode the yuv file on the ics and omap4. the omx_h264enc can encode the file but the result file can not play by the player on pc. and i find there is no sps and pps in this file . alsa i find the omap4 colorformat is OMX_TI_COLOR_FormatYUV420PackedSemiPlanar. i want to set the colorformat of the encoder to OMX_COLOR_FormatYUV420PackedSemiPlanar ,but i failed i read colorformat of the in port also OMX_TI_COLOR_FormatYUV420PackedSemiPlanar . Can this omap4 only support the OMX_TI_COLOR_FormatYUV420PackedSemiPlanar colorformat ?If that is true , i want to give the OMX_TI_COLOR_FormatYUV420PackedSemiPlanar to encoder . What format of yuv about the OMX_TI_COLOR_FormatYUV420PackedSemiPlanar ?

the code is below:

                G_OMX_PORT_GET_DEFINITION (omx_base->in_port, &param);
               dumpPortStatus(omx_base->in_port);/*****ADD******/
                width = param.format.video.nFrameWidth;
                height = param.format.video.nFrameHeight;
                framerate = param.format.video.xFramerate;
               param.format.video.nStride = 1280;
               param.format.video.eColorFormat = OMX_COLOR_FormatYUV420PackedSemiPlanar;
               /* this is against the standard; nBufferSize is read-only. */
                fourcc = g_omx_colorformat_to_fourcc (param.format.video.eColorFormat);
                param.nBufferSize = gst_video_format_get_size_strided (
                        gst_video_format_from_fourcc (fourcc),
                        width, height, param.format.video.nStride);
                error_val = G_OMX_PORT_SET_DEFINITION (omx_base->in_port, &param);
               dumpPortStatus(omx_base->in_port);

Thank you very much!

over 13 years ago

0 Manuel Contreras over 13 years ago

TI__Genius 10545 points

In a test way you can use code for example from Android source code in 4AI1.4.P1,

http://git.omapzoom.org/?p=platform/frameworks/base.git;a=blob;f=media/libstagefright/OMXCodec.cpp;h=60d9bb727800c4a6c56fbdbe6ef889fc9700aae7;hb=refs/heads/master

./mydroid/frameworks/base/media/libstagefright/OMXCodec.cpp

status_t OMXCodec::isColorFormatSupported(
OMX_COLOR_FORMATTYPE colorFormat, int portIndex) {

and check that you are setting correct index in your code, and some variables seems to be copied at the other side, try to put them in left side

you could try to use next function's code as reference, it is in the same file

void OMXCodec::setVideoInputFormat(
const char *mime, const sp<MetaData>& meta) {

and in case it doesn't fix the issue, could you add some logs about actual error to check what error is being returned? and you are using Android, what release are you using? can you provide code at the surroundings of shared code? like code where port index is added before calling OMX_GetParameter.

check http://e2e.ti.com/support/omap/f/849/t/179160.aspx for more information about what posts must contain.

You can use get some debug prints using information from next link

http://omapedia.org/wiki/Debugging_RPMsg

0 Aaron Pope over 13 years ago in reply to Manuel Contreras

Intellectual 465 points

Hi, Manuel:

Thank you for your answer. The version of ics is 4.0.3。 The gstreamer command is "gst-launch -v filesrc location=/mnt/ext_sdcard/nv12.yuv ! videoparse width=1280 height=720 ! omx_h264enc ! filesink location=/mnt/sdcard/test1.264 --gst-debug=omx:5". I use the ./mydroid/frameworks/base/media/libstagefright/OMXCodec.cppstatus_t OMXCodec::isColorFormatSupported(OMX_COLOR_FORMATTYPE colorFormat, int portIndex) it can support the color format of OMX_TI_COLOR_FormatYUV420PackedSemiPlanar . but it is not support OMX_COLOR_FormatYUV420PackedSemiPlanar.so i can not understand .I use the movie editor apk and it can encode mp4 file successful. the movie editor also use openmax .and in the source code hardware/ti/domx/omx_proxy_component/omx_h264_enc/src/omx_proxy_common.c

OMX_ERRORTYPE LOCAL_PROXY_H264E_SetParameter(OMX_IN OMX_HANDLETYPE hComponent, OMX_IN OMX_INDEXTYPE nParamIndex, OMX_IN OMX_PTR pParamStruct)

if(nParamIndex == OMX_IndexParamPortDefinition)
   {
       pPortDef = (OMX_PARAM_PORTDEFINITIONTYPE *)pParamStruct;
       if(pPortDef->format.video.eColorFormat == OMX_TI_COLOR_FormatYUV420PackedSemiPlanar)
       {
           pPortDef->format.video.eColorFormat = OMX_COLOR_FormatYUV420PackedSemiPlanar;
       }
#ifdef ANDROID_CUSTOM_OPAQUECOLORFORMAT
       else if(pPortDef->format.video.eColorFormat == OMX_COLOR_FormatAndroidOpaque)
       {
           if(COLORCONVERT_open(&pProxy->hCC,pCompPrv) != 0)
           {
               PROXY_assert(0, OMX_ErrorInsufficientResources,
                           "Failed to open Color converting service");
           }
           pProxy->bAndroidOpaqueFormat = OMX_TRUE;
           pPortDef->format.video.eColorFormat = OMX_COLOR_FormatYUV420PackedSemiPlanar;
       }
#endif

I think it will covert OMX_TI_COLOR_FormatYUV420PackedSemiPlanar to OMX_COLOR_FormatYUV420PackedSemiPlanar. Is it effect? and i set the parameter

                param.format.video.eColorFormat = OMX_COLOR_FormatYUV420PackedSemiPlanar;

               /* this is against the standard; nBufferSize is read-only. */
                fourcc = g_omx_colorformat_to_fourcc (param.format.video.eColorFormat);
                param.nBufferSize = gst_video_format_get_size_strided (
                        gst_video_format_from_fourcc (fourcc),
                        width, height, param.format.video.nStride);
                error_val = G_OMX_PORT_SET_DEFINITION (omx_base->in_port, &param);
               GST_DEBUG_OBJECT(omx_base," error_val = %x\n",,error_val);/*****ADD******/
               dumpPortStatus(omx_base->in_port);

    the log is

Input Port

eCompressionFormat = OMX_VIDEO_CodingUnused

eColorFormat = OMX_TI_COLOR_FormatYUV420PackedSemiPlanar

so the eColorFormat is wrong .

omx_proxy_common.c

OMX_ERRORTYPE PROXY_AllocateBuffer(OMX_IN OMX_HANDLETYPE hComponent,
    OMX_INOUT OMX_BUFFERHEADERTYPE ** ppBufferHdr,
    OMX_IN OMX_U32 nPortIndex,
    OMX_IN OMX_PTR pAppPrivate, OMX_IN OMX_U32 nSizeBytes)

if (nStride == LINUX_PAGE_SIZE && \
pCompPrv->proxyPortBuffers[nPortIndex].proxyBufferType != EncoderMetadataPointers) //Allocate 2D buffer

typedef enum PROXY_BUFFER_TYPE
   {
       VirtualPointers,   /*Used when buffer pointers come from the normal A9 virtual space */
       GrallocPointers,   /*Used when buffer pointers come from Gralloc allocations */
       IONPointers,       /*Used when buffer pointers come from ION allocations */
       EncoderMetadataPointers,       /*Used when buffer pointers come from Stagefright in camcorder usecase */
       BufferDescriptorVirtual2D          /*Virtual unpacked buffers passed via OMX_TI_BUFFERDESCRIPTOR_TYPE */
   } PROXY_BUFFER_TYPE;

My buffer is not from stagefright , so i think proxyBufferType should be VirtualPointers. Is that right? if it is right why the judge above have no effort?

Further more,the result 264 file after the command has not sps/pps, Is the state of omx_h264enc right?even the colorformat is wrang , the sps/pps should in the 264 file.

Thank you very much!

0 Chintan Patel over 13 years ago in reply to Aaron Pope

TI__Intellectual 1410 points

Aaron,

Encoders on OMAP4 supports OMX_TI_COLOR_FormatYUV420PackedSemiPlanar as input format only. This format refers to the NV12 buffer holding Y an UV buffers in contiguous memory area.

Regards,

Chintan

0 Manuel Contreras over 13 years ago in reply to Chintan Patel

TI__Genius 10545 points

Aaron;

I check the code and there is not clear why code is changing it to non TI index when it is passed Ducati side. I imagine that if you use OMX_TI_COLOR_FormatYUV420PackedSemiPlanar your code works and it is not needed to use the other index or to specify it, can you confirm this observation?

I have read the code for AFS and Video encode-decode code, they do some check up about this OMX_TI_COLOR_FormatYUV420PackedSemiPlanar value in some places before reaching the OMX_StateIdle state, that could be the reason why it is not taking affect the use of OMX_COLOR_FormatYUV420PackedSemiPlanar that is assigned to Port definitions structure afterwards.

About

if (nStride == LINUX_PAGE_SIZE && \
pCompPrv->proxyPortBuffers[nPortIndex].proxyBufferType != EncoderMetadataPointers) //Allocate 2D buffer

Can you check comments made in

http://e2e.ti.com/support/omap/f/849/p/192468/691188.aspx

this ti check if proper path is open for code selection.

About buffer size, DOMX has specific buffer requirements then they are calculated internally and if needed some change it is communicated using port reconfiguration event.

Two situation I know you could use to modify buffer size, is 1. if you plan to use a METADATA structure added to your buffer and 2. you need to reduce input buffer size for video encode, actual value is worst case scenario to use and to avoid reallocating buffers if input stream has a frame bigger than current size.

About SPS+PPS you posted it in another post

http://e2e.ti.com/support/omap/f/849/p/193242/691276.aspx

But here are those answers:

1. Can you do a buffer dump at OMX_FillBufferDone to check if SPS+PPS are being send from video encoder, you can dump only first 32 bytes that must be enough to see SPS+PPS and initial headers for first IFrame. This to discard failed areas or isolate the issue to one area.

2. If you are using a full pipe line to encode-decode process and then to a check, if encoder is generating the SPS+PPS they are stored in GST decoder in a variable like codec data, for what I know, I don't have much data about your actual GST, it is required to read this variable when decoding process is started and send it before sending the first Iframe.

It is needed to check full configuration and possibly to check some logs for something that could be wrong before these unsuccessful calls.

if (nStride == LINUX_PAGE_SIZE && \
pCompPrv->proxyPortBuffers[nPortIndex].proxyBufferType != EncoderMetadataPointers) //Allocate 2D buffer

0 Aaron Pope over 13 years ago in reply to Manuel Contreras

Intellectual 465 points

Hi！

I dump the buffer at OMX_FillBufferDone and find the sps+pps in the 264 file produced by ics. I have run the command on the android 2.3 and omap4 .the 264 file produced by 2.3 contain sps+pps .the sequence is 00 00 00 01 27 and 00 00 00 01 28 .the gst-openmax source code on ics 4.0.3 has gotten from the android 2.3. The 264 file produce by ics do not have the sps+pps at the beginning of the file , at that location is 00 00 00 01 25 this is IDR. If i use the gstreamer filesrc the output port proxyBufferType is VirtualPointers. so the omx_h264enc can run on the ics 4.0.3 and omap4 . but the content of pictures which of video file produced by encoder are error.for example , two neighbouring frames has encoded into one picture or the chroma in the pictures are wrong .but the first picture is right ,both luminance and chroma.so I think if the color format is wrong,the Y will be right but the UV must be wrong in all of picture.i think this phenomenon is about the synchronization . is it right? what configuration will influence the omx_h264enc produced sps+pps?

Thank you!

0 Manuel Contreras over 13 years ago in reply to Aaron Pope

TI__Genius 10545 points

You can check next file for OMX_VIDEO_PARAM_AVCNALUCONTROLTYPE

./4ai/4AI.1.1/mydroid/hardware/ti/domx/omx_core/inc/OMX_TI_Video.h

It needs to be set when OMX_StateLoaded with OMX_SetParameter.

Read .h file for more details about it's use.

About incomplete pictures, try to check input buffer contents.

0 Manuel Contreras over 13 years ago in reply to Manuel Contreras

TI__Genius 10545 points

In my previous post I referenced code to 4AI1.1 but it is present actually in latest 4AI1.5 too. Please refer to 4AI1.5 code instead.

0 Aaron Pope over 13 years ago in reply to Manuel Contreras

Intellectual 465 points

Hi,

now the luminance and chrome of 264 file are all right. because i expand the width of input yuv buffer in the videoparse plugin to 4096 but not notice the offset , so the picture are wrong.the color format is OMX_COLOR_FormatYUV420PackedSemiPlanar.

Thank you very much !

Br,

yangliming

Processors

Processors forum

gst-openmax and openmax on ics and omap 4