8127 [IPNC RDK V2.8] resolution and h264 profile question

My experience with RDK is that many parts are hard coded, or using a macro. You have to dig deep yourself to find exactly what's wrong.

the assertion failure, since you already know where it fails, you can easily trace back the root cause, I guess.

Hi Pei,

Can you try disabling the MV sad ? Probably you can set "enableAnalyticinfo" flag to zero in the usecase file.

I can't find the root cause, where the "minOutBufSize[1].bytes" was decided. I guess it was in the h264_enc lib.

Actually I shield this UTILS_assert() there, and we can get 720p thus. 

but it was not the final proper method.

Hi Radhesh

I'll try, and I wonder which aspect the MV sad effect, 720P or h264 profile, or both?

And, disable it, means disable the video analytics algorithm on dsp?

Many thanks!

Jacky Pei.

Hi Pei,

Not the analytic algo on DSP. H264 lib can generate analytic info (MV and SAD) which you can control at create time by setting the create time parameter "enableAnalyticinfo" flag.  YOu can find the flag in McFW and set it to "0".

Hi, Radhesh

I found some clue from <H264_Encoder_HDVICP2_UserGuide.pdf>, that

"IH264ENC_PROFILE_INCOMPLAINT_TRANSFORMBLOCKSIZE = 3" is

"This error is set when  IH264ENC_Params::transformBlockSize != IH264_TRANSFORM_4x4 && IVIDENC2_Params::profile != IH264_HIGH_PROFILE. "

 and "IH264ENC_PROFILE_INCOMPLAINT_SCALINGMATRIXPRESET = 5" is

"This error is set when scaling matrix is used without IH264_HIGH_PROFILE."

Thus, I modified below code:

========code========

encLink_h264.c

line:653:  staticParams->transformBlockSize = IH264_TRANSFORM_4x4;//IH264_TRANSFORM_8x8 ;

line:661:  staticParams->rateControlParams.scalingMatrixPreset =

                        IH264_SCALINGMATRIX_NONE;//IH264_SCALINGMATRIX_NORMAL;

line:738: staticParams->transformBlockSize = IH264_TRANSFORM_4x4;//IH264_TRANSFORM_8x8 ;

line:752:  staticParams->transformBlockSize = IH264_TRANSFORM_4x4;//IH264_TRANSFORM_8x8 ;

line:1080:  dynamicParams->rateControlParams.scalingMatrixPreset =

                        IH264_SCALINGMATRIX_NONE;//IH264_SCALINGMATRIX_NORMAL;

========code========

but there is also error:

[m3video]  11110: ENCODE: Creating CH0 of 1920 x 1080, pitch = (2048, 2048) [PROGRESSIVE] [NON-TILED  ], bitrate = 2000 Kbps ...

[m3video] 11215:!ERROR!:ENCLINK::links_m3video/iva_enc/encLink_h264.c:[195]::INTERNAL ERROR:-1  

[m3video] ALGCONTROL FAILED:CMD:6  [m3video] ERROR: XDM_UNSUPPORTEDPARAM = 14,  

[m3video] ERROR: XDM_FATALERROR = 15  

[m3video] ERROR: IH264ENC_UNSUPPORTED_INTRACODINGPARAMS = 23,  

[m3video] 11216:!ERROR!:ENCLINK::links_m3video/iva_enc/encLink_h264.c:[195]::INTERNAL ERROR:-1  

[m3video] ALGCONTROL FAILED:CMD:9  [m3video] ERROR: XDM_UNSUPPORTEDPARAM = 14,  

[m3video] ERROR: XDM_FATALERROR = 15  

[m3video] ERROR: IH264ENC_UNSUPPORTED_INTRACODINGPARAMS = 23,  

[m3video] 11216:!ERROR!:ENCLINK::links_m3video/iva_enc/encLink_h264.c:[195]::INTERNAL ERROR:-1  

[m3video] ALGCONTROL FAILED:CMD:3  [m3video] ERROR: XDM_UNSUPPORTEDPARAM = 14,  

[m3video] ERROR: XDM_FATALERROR = 15  

[m3video] ERROR: IH264ENC_UNSUPPORTED_INTRACODINGPARAMS = 23,  

[m3video]   

[m3video] 11217:ERR::linkID:10000024::channelID:0::errorCode:-5::FileName:links_m3video/iva_enc/encLink_h264.c::linuNum:1336::errorCondition:(algStatus == XDM_EOK)  

[m3video] ENCLINK_H264:HEAPID:0        USED:1896  

[m3video]  11218: Assertion @ Line: 907 in links_m3video/iva_enc/encLink_common.c: retVal == ENC_LINK_S_SUCCESS : failed !!!  inside autorun INIT: Entering runlevel: 5 

About "ERROR: IH264ENC_UNSUPPORTED_INTRACODINGPARAMS = 23", the <userGuide> describe it as "This error is set when any parameter of structure IH264ENC_IntraCodingParams is not in allowed range."

I don't know how to deal with it!

I also disable the enableAnalyticinfo:

App_runDemo():   

//vencParams.encChannelParams[0].enableAnalyticinfo = 1;   

vencParams.encChannelParams[0].enableAnalyticinfo = 0;

and the error is same with above.

How can I deal with this, I need an exact statement to set the stream to Main profile and BP.

Thanks and regard.

Jacky Pei.

Dear Radhesh

We update the SDK to Ver 3.2, and it support High profile, MP and BP well now.

But I still met some difficult  changing the resolution.

If I change the D1 to CIF, error occurs in the display link:

thus I also removed the TVOUT link, and there seemed some error in [host] as below:

[host]  SYSTEM: Invalid proc ID ( procID = 15, linkID = 0xffffffff, cmd = 0x00004004)  

ASSERT (system_linkApi.c|System_linkControl|42)

TimeOut occure in boot_proc.

Program exit.

TimeOut occure in boot_proc.

Program exit.

I really have no idea that how to work with it smoothly!

I attachmed my modified code and the serial information, would you kindly give me some suggestion that how can I change the resolution?

4544.multich_tristream_fullfeature.c

1803.CIF.log

Regards!

Jacky.

Hi, Radhesh

The display link error is because I didn't clean up all the display link, and CIF Resolution works as I modified the code:

int Init_StreamEnv(StreamEnv_t* pConfig)

{

    // SetDisplayValue(pConfig -> nDisplay);

}

Now, the problem is only about 1080p to 720p, as I mentioned before.

1080P_H264 changed to 720P_H264,Error occurs in UTILS_assert():

          encLink_h264.c

          enclink_h264_set_algObject()

          {

               ........

               UTILS_assert((status->videnc2Status.bufInfo.minOutBufSize[i].bytes < algCreateParams->mvDataSize));

          }

         I print those variables:

         "minOutBufSize[1].bytes = 148172", which is the same with 1080P condition,

         "algCreateParams->mvDataSize = 86400", which is 194400 in 1080P condition.

        Thus, it seems that the "minOutBufSize[2].bytes" remains the same ,which should be changed according the resolution.

Thought if shield the UTILS_assert, 720p could be gotten, but I don't think it's a proper method.

Could you help us analysis this problem?

Thanks!

Jacky.

Hi Pei,

Good to know that you were able to solve the CIF issue. 

Can you check if you are using latest version of H264 encoder? I suspect some issue with the code (i see some problem in the older version of the codec)

Hi, Radhesh

In the path: IPC3.2\Source\ti_tools\codecs-dm814x\REL.500.V.H264AVC.E.IVAHD.02.00.04.00\500.V.H264AVC.E.IVAHD.02.00

<H264_Encoder_HDVICP2_ReleaseNotes.pdf>, it shows "Build ID: 02.00.04.00", "September 2012".

I think it would be the latest version.

Regards!

Jacky Pei

Hi Pei,

Disabling "enableAnalyticinfo" did not help you?

Hi, Radhesh

thanks for your help.

In version 3.2, Disabling "enableAnalyticinfo" helps with the below assert error:

enclink_h264_set_algObject()

{

     UTILS_assert((status->videnc2Status.bufInfo.minOutBufSize[i].bytes < algCreateParams->mvDataSize));

}

but, there's some new problems if changed 1080p t0 720p:

1. In IE, we can't get this 720p h264 stream, on this stream's box shows black;

2. RTSP didn't run for this stream:

    we can get D1 stream through port 8556 via VLC, but we can not get the 720p, we found that 8556 port is listened while 8557 port is not. 

    net satat shows below:

    root@DM8127_IPNC:~# netstat -tnl | grep 8556

    netstat: /proc/net/tcp6: No such file or directory

    tcp        0      0 0.0.0.0:8556            0.0.0.0:*               LISTEN     

    root@DM8127_IPNC:~# netstat -tnl | grep 8557

    netstat: /proc/net/tcp6: No such file or directory

    root@DM8127_IPNC:~#

 3. we get h264 720p stream data via circular buf, and simply play it with the Elecard, some error occurs while playing.

 4. We modified the resolution at the camera output, thus the MJPEG stream changed to 720p also.

     we saw this stream abnormal on IE, and on the serial port print many "Failed to find SOF0 marker in JPEG header!"

     the IE shows like this:

The black box is h264 720p stream, the right top one is the h264 D1 stream, and the lower right one is the Mjpeg 720p stream.

 Attach is the serial port information.

6320.720P_c.log

Regards.

Jacky

Hi Jacky,

While I look into this, can you check if 720P video output at HDMI is proper?

HI Jacky,

I have one more suggestion. In the current IPNC, there seems a support for OV sensor with 720P. Probably you can check for IMGS_OMNIVISION_OV10630 in entire MCFW and see whether you have taken care all the necessary changes w.r.t. resolution.

Hi, Radhesh

Thanks for your help.

I test the 720P on HDMI, It's ok.

The HDMI's data is from DupLink, which is previous of the encLink, so this is check camera output data, is it?

And the sensor is MT9J003, not match OV10630, is it proper to define the MACRO IMGS_OMNIVISION_OV10630?

Actually, besides of the resolution, the difference of define this MACRO or not is

pCameraOutPrm->dataFormat = SYSTEM_DF_YUV422I_VYUY; // OV10630

pCameraOutPrm->dataFormat = SYSTEM_DF_BAYER_RAW; // others

and use of  ispLink.

Regards.

Jacky.

Hi Jacky,

Do not define the macro IMGS_OMNIVISION_OV10630 and also do not change data format as well. What I am suggesting is just check for changes done for only width and height. Currently you would have changed width and height to 1280 x 720 for MT9J003, but have you done changes in all the necessary places? 

Hi, Radhesh

You mean that take the OV10630 as a reference to check the necessary place to be changed.

I think I've changed the according place.

Now, I've done 3 tests:

1. change the camera resolution 1080p to 720p, then both the h264 1080p and mjpeg 1080p will changed to 720p,

    that failed as I mentioned before;

I also did below change, by adding scaler link,to change those resolution separately, and that was succeed.

2. Add scaler link between DUP2_0 and MERGE to change h264 1080p to 720p, succeed;

3.Add scaler link between DUP2_1 and MERGE to change mjpeg 1080p to 720p, succeed.

I wonder why change both the two stream failed while succeed changing it separately. 

Could you help me analysis this, and some suggestions to help me understand the MCFW and Links more deeper?

Attach is my changed code, you need modify the suffix to c.

multich_tristream_fullfeature.c.cam720p :  Modify the resolution at camera output;

multich_tristream_fullfeature.c.cif : Modify D1 to CIF;

multich_tristream_fullfeature.c.h2720p : Modify h264 1080p to 720p by adding scaler link;

multich_tristream_fullfeature.c.mj720p : Modify mjpeg 1080p to 720p by adding scaler link;

Thanks, Radhesh.

Regards.

Jacky.

Hi Jacky, I would recommend you not to share source code in the public forum. Probably we can take it offline. You can send email to radhesh.bhat@pathpartnertech.com. BTW, I am on business travel till 4th from today. I would look into this once I am back. Hope that is fine with you.

Ok, Radhesh

I removed the attach code above, and send it to your email.

Hope our going on when you back and wish you a good travel.

Regards.

Jacky.

Hi, Radhesh

when I changing D1 to CIF, there are colored lines under the OSD text.

It occurs in both the two resolution changing methods: modify camera output or add scaler link.

And, the lowercase and number have lilac underline, capital has green underline.

How could I deal with this?

Regards.

Jacky.

Hi Jacky,
Help me to understand your issue with more details.

1. I assume you dont see this issue at D1. Is that correct?
2. I assume the OSD texts are custom defined by you.
3. What is the color format are you passing to OSD ? does it match with the color format of your image?
4. What is the resolution of your OSD text? is width and height are non multiple of 2?

Hi, Radhesh

1. I assume you dont see this issue at D1. Is that correct?

    Jacky>> This is seen only in CIF

2. I assume the OSD texts are custom defined by you.

    Jacky >> I use the default set, this is set in the IE -> Settings -> Video -> Stream2 -> Overlay setting.

3. What is the color format are you passing to OSD ? does it match with the color format of your image?

4. What is the resolution of your OSD text? is width and height are non multiple of 2?

    Jacky >> In the fullfeature usecase, only a few OSD param could be set, such as prevLinkId and nextLinkId, Where to set the color format and resolution of OSD?

              >> I think I used the same param as in D1 resolution.

              >> I found function: Vsys_setSwOsdPrm()，and the enum VSYS_SWOSD_SETPARAM to set params as framerate, bit rate..., no resolution and color format param found. Need I set those params according to the stream resolution? where to set it?

Thanks.

Jacky.

For point 4, you can cross check on the parameters under ..\ipnc_rdk\ipnc_mcfw\mcfw\src_bios6\alg\swosd\src

and ..\ipnc_mcfw\mcfw\src_bios6\links_c6xdsp\alg_link

Hi, Radhesh

I've beening looking into the OSD code now.

The width and height trans to OSDLink creating function and windows creating function (DM81XX_SWOSD_setWinParam()) are right.

and, In CIF resolution, the Date and Time Info OSD is ok,

which using the same parameter with the user string and detailed Info OSD.

thus I think the OSD window is fine. Maybe the bug is in FontDB, or the UV components?

I checked the height and width of OSD windows is multiple of 2, and I will check the fontDB.

Regards

Jacky.

Hi, Radhesh

I found some clue about the OSD problem.

It's in the function SWOSD_MakeOsdwinstring()'s first argument, SWOSD_Fontdata.

The function converts input string  to  YUV  data, and update the OSD bitmap window width, height and line offset.

if the SWOSD_Fontdata use below setting, there is no underline,

  fontInfo.fontAddr = fontBuff462_Y;        

  fontInfo.fontAddrUV = fontBuff462_UV;        

  fontInfo.fontDB_Y = fontDB462_Y;        

  fontInfo.fontDB_UV = fontDB462_UV;

and if use those param below, there comes out the underline:

  fontInfo.fontAddr = fontBuff462_Y1;        

  fontInfo.fontAddrUV = fontBuff462_UV1;        

  fontInfo.fontDB_Y = fontDB462_Y1;        

  fontInfo.fontDB_UV = fontDB462_UV1;

The difference is as below :

fontBuff462_Y copys from YData_462_12,

fontBuff462_UV copys from UVData_462_12;

fontBuff462_Y1 copys from YData_462_12_WFONT,

fontBuff462_UV1 copys from UVData_462_12_WFONT.

You can find those in ../ipnc_mcfw/mcfw/src_bios6/links_m3vpss/alg/sw_osd/ti_swosd.c in func DM81XX_SWOSD_create().

Could you tell me what's the difference between YData_462_12 and YData_462_12_WFONT, UVData_462_12 and UVData_462_WFONT?

ps: They are defined in  ../ipnc_mcfw/mcfw/src_bios6/links_m3vpss/alg/sw_osd/fonts/ascii_UV_462_12.c and ascii_Y_462_12.c.

Besides, the OSD's in CIF is not as clear as in 720P and 1080P. Do you have suggestions to those problems?

Thans and regards.

Jacky.

Hi Jacky,

When you say,

Pei Jacky said:

if the SWOSD_Fontdata use below setting, there is no underline,

  fontInfo.fontAddr = fontBuff462_Y;        

  fontInfo.fontAddrUV = fontBuff462_UV;        

  fontInfo.fontDB_Y = fontDB462_Y;        

  fontInfo.fontDB_UV = fontDB462_UV;

is this working for CIF ? right?

Pei Jacky said:

Could you tell me what's the difference between YData_462_12 and YData_462_12_WFONT, UVData_462_12 and UVData_462_WFONT?

They actually differ w.r.t. to the transparency feature supported in the OSD library.

Can you confirm the OSD mode you are using? (check for "OSD_MODE" in the Rules.make

Hi, Radhesh

I found no OSD_MODE setting in the Rules.make.

The "fontBuff462_Y, fontBuff462_UV, fontDB462_Y, fontDB462_UV" and  "fontBuff462_Y1, fontBuff462_UV1, fontDB462_Y1, fontDB462_UV1"are working for CIF.

There defined three macro:

#define VIDEO_RESOL_WIDTH_LARGE         1000

#define VIDEO_RESOL_WIDTH_MEDIUM        750

#define VIDEO_RESOL_WIDTH_SMALL         400

If the width is small then VIDEO_RESOL_WIDTH_SMALL, the *462_* will be used.

and, if defined SWOSD_USER_TRANSPARANCY_ON, the "fontBuff462_Y1,..." will be used for DATE and TIME Info, other OSD use "fontBuff462_Y,...". that's why in the first picture I attached, DATE and TIME OSD have no underlines. So I think the fontBuff462_Y1 may not be working finely.

Regards.

Jacky.

HI Jacky, If you have referred to the section 5.10 of SWOSD userguide,it is mentioned abt constraints on the Fixed transparency. Check if that matches your observation.

Hi, Radhesh

In user guide, it says that "the OSD  bitmap  window  width  is  multiple  of  4" and "YUV420" should be matched in condition of Fixed transparency.

In CIF, the OSD bitmap window width is defined in function DM81XX_SWOSD_setWinParam() in ../mcfw/src_bios6/link_m3vpss/alg/ti_swosd.c

bmpWinPrm.width = SWOSD_STRINGWIDTH_462_12;

the value is 462, not multipul of 4.(The macro is defined in swosd_priv.h).

The format is SWOSD_FORMATYUV420p, matches the "YUV420".

I'm confused that:

1. DATE and TIME Info OSD use the same bmpWinPrm.width with Detailed Info and User string, why DATE and TIME OSD has no underline but others have?

2. I modified the SWOSD_STRINGWIDTH_462_12 to 464, the OSD is like this:

 So, How or if can I modify the SWOSD_STRINGWIDTH_462_12?

Thanks for you suggestions and help.

Regards.

Jacky.