need IPNC 8127 frame rate control link

Hi Changchun,

This isn't an answer. But you have asked several questions on the E2E forums regarding the DM8127 IPNC RDK that I also wanted to ask. So Thanks! for that.

I agree that there should be "Frame Rate Control Link" as part of the McFW. I need this feature for my usecase but for now I'm just setting the frame rate from the web interface (which changes it at the Camera/Capture Link source). I'd rather be able to change the frame rate at a later / lower point on a single specific Chain using a "Frame Rate Control Link". So yes, I have the same question for TI.

I've also experimented with the Scaler Link but it only seems to work in very limited positions in a Chain (unfortunately not in the Chain position that I would like to use for my usecase). I would like to see it improved if possible. Right now the only way I know to change the resolution is to modify code in the ISS Capture driver file, "issdrv_captureApi.c"(Marko Shink showed me how to do that, Thanks! Marko). I think I saw a reply to one of your other questions (http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/716/t/181831.aspx#662760) that said the soon-to-be-released G.A. Release 2.8.0 will make changing resolutions easier. But that will only change the resolution at the Camera source (I think). I still would like to see the Scaler Link improved so the resolution could be easily changed in the middle of a Chain.

Also more documentation for usage of all the Links / Chains API and McFW API would be nice.

Thanks for listening!

Allen

Hi Changchun & Allen,

can you describe your data chain and explain where you need an individual and dynamic framte rate control?

We don't have specific frame rate control link in RDK because a lot of links (camera, encoder, etc) internally supports the frame rate control function. Except for the camera link where we can change the sensor output settings, frame control on other links is simply implemented with a frame skipping mechanism, it should be easy for you to port this frame skipping function to any link you want.

For example, you can look in nsfLink_drv.c how we call Utils_doSkipFrame to conrtol the output frame rate.

Hi Zhuoran,

Thanks for your reply.

My data chain is like this:

From the camera link, one path is for recording, which is H.264 format (H.264 1080P + H.264 720P). The other path goes for video analytics. Since the video analytics cannot handle 60fps 1080P, we need to downsample to 384x216 through a scaler link (this is well done). After that, we follow the DSP usecase for in-place processing with osd link on DSP. However, even 384x216@60fps is still too many data to handle by DSP, which can only process around 30fps and leave the others unprocessed. Therefore, we need a general frame rate control link to limit the frame to handle. However, we cannot use the frameskipping method you suggested, because we want to handle every frame in the path, instead of skipping part of them. Otherwise, you will see some frames are processed, while others are not processed in the display of MJPEG output.

If I need to write such a frame rate control link, what files and functions should I change? For example if I define a new VPSS_LINK(62) for the frame rate control link, do I need to rebuild syslink in ti_tools\ folder? Is 62 a good number (I see VPSS_LINK(61) is in use). Should it be running on m3vpss or other place? I saw scaler and DUP are running on m3vpss in the DSP usecase.

By the way, in nsfLink_drv.c, I don't find a function named Utils_doSkipFrame. I am using IPNC_RDK 2.0.0.

Thanks,

Changchun

Hi Changchun,

Utils_doSkipFrame is not defined but called in nsfLink_drv.c, can you locate it? I just want to show you an example how we use it. Otherwise you can search it in ipnc_mcfw.

When you decide to skip one frame in a link, you have two options, either you send it directly to the ouput queue (in this case the frame rate will not change and like you said some frames are processed but others unprocessed in the output stream), either you can release it back to the previous link (in this case the output frame rate will be changed). Actually in our RDK while we do frame skipping, only the second option is used. For example if you have 60fps in input and you decide to process only frame 0, 2, 4... and immediately release frame 1, 3, 5 to the previous link, then you only output half of frames and get 30fps. Can you understand this?

I guess you video analytics data chain should be like this, right?

camera[1](702P) => dup(720P) => scaler(384*216) => DSP => MJPEG encoding

                                                   =>H264 encoding

Then the best way to do the frame skipping is inside the sclar link, again you really don't need to create a new link. 

Hi,

Interesting. We though of the solution together. I just implemented the frame skipping inside scaler link this afternoon. However, I still feel a general frame rate control is necessary, since we might need to do so on some position without scaling. So, if I want to create such a link, where should I refer to? Any link api writing reference?

Changchun

We don't have such reference because we don't recommand customers to create their own links. However, if you really want to do it, you can just take a simple link (like dup, merge) as a starting point and try to modify it.

Again, the frame skipping is a very simple function that you can integrate in every link. A new link means a new thread and one more message for each frame exchange, which is a wasting of resource if just using for skipping frames.  

Zhuoran Dai said:

We don't have such reference because we don't recommand customers to create their own links. However, if you really want to do it, you can just take a simple link (like dup, merge) as a starting point and try to modify it.

Again, the frame skipping is a very simple function that you can integrate in every link. A new link means a new thread and one more message for each frame exchange, which is a wasting of resource if just using for skipping frames.  

Frame control is only one customized link example. If the end user is allowed to add more links, they can design more custom links to meet the needs. Otherwise, the link itself is far from enough to meet the variety of different customuers, especially those who want to do more work with the computing power on camera. I know TI has the c6run and c6app utility, but they are too simple by just replacing some function calls or library with DSP, a system level developer needs more control over the DSP, such as how to share memory, communicate between each other. All these can only be done through the link under current framework.