IPNC links repeating same images

Hi Karsten,

Can you move to the latest versions? The latest version is 3.8.

I think 2.8 is very old and there have been many updates after that release.

Regards

Rajat

Hi Rajat,

It is unfortunately not an option to port all our changes from IPNC 2.8 to version 3.8 because we have made a lot of extensions to the MCFW and ISS (as well as the PSP).

However I would like to get your opinion on what could be causing this kind of behavior. Please remember that this is something that happens after a while (not when I start up the camera) and it started to happen when I added more traffic to the ISS bus.

If I capture a raw image directly from the VP then the image is "new" and correct. But the output from both Resizers seems to be the same old images being repeated over and over.

So basically it looks like the CameraLink starts to output images to the rest of the links which are old - but the ISS works correctly.

Where are the old images stores, and how come that the images gets recycled through the system? I would like to know a way for me to debug this further and see if there is a good explanation and a fix to this very bad issue.

Best regards,

Karsten

Hi Rajat,

I got a bit more information about this problem because I added some debug output in the iss capture driver.

When this problem happens then I can now see that I am not getting the MSP_ISS_DRV_RSZ_END_ISR interrupts any more - but I keep getting the MSP_ISS_DRV_ISIF_VD_ISR.

So does that actually mean that the resizer(s) are stuck and needs to be software reset in order to get the images back?

PS:

I saw that there is a procedure on how to reset the resizers in the ISS manual - but I would prefer that the resizers doesn't get stuck in the first place. Could you perhaps tell me what could cause this to happen in the first place - and what can be done in order for me to guard against this problem??

Best regards,

Karsten

resizer hang is a well known issue. check out this thread:

http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/716/t/163881.aspx

there are many other related threads too.

I ended up changing our DDR connection from EMIF0 only to both EMIFs, while keeping 32bits interface.
there's no bandwidth increase, but the problem is gone. it must be related to the DDR bw scheduling.

Hi Hongfeng,

Thank you for the reply.

We already use both EMIF's in our design and that is probably why we can achieve a higher frame-rate before we see this problem.

But the link actually helped because I tried to add more frame-blanking time to the ISS bus transfer and this looks like it fixes the problem. I need more testing before I can confirm 100% that the resizer doesn't stop anymore but so far so good.

Best regards,

Karsten

Karsten,

We had pushed some performance related fixes in v.3.8 which will also help you.

I see that you already tried increasing blanking and that does help.

Also, you can try increasing the ISS DMA output priority to the L3 interconnect. The method is documented int he attached document.

Regards

Rajat

Hi Rajat,

Thank you for the follow up on this matter.

I have downloaded the v.3.8 of IPNC so I will consider back-porting some of the performance enhancements to our version.

Could you point me in the right direction on where the RSZ stall detection and automatic reset code is located in the sources (in the ISS I assume) then I will take a closer look?

I read the attached document and I am convinced that we have seen similar issues as described where the ARM depletes the DDR resources so the ISS DMA transfer will slow down too much. I will try to implement the recommended priority changes.

PS:
Are there any bug fixes or performance improvements in the MJPEG and/or the H.264 Encoder modules that I can utilize?

Best regards,

Karsten

Hi Rajat,

Where do I correctly implement the ISS BW regulator settings and the PBBPR register settings? Is that done in the PSP (board initialization in ./kernel/arch/arm/mach-omap2/) or in the UBoot initialization?

And does the kernel/UBoot code from v3.8 already contain this kind of prioritization code?

Best regards,
Karsten

Hi Rajat,

I never saw any answers to my questions about where to add the BW regulator settings and about how to reset the stalled RSZ! It has now been nearly two months since I posted these questions so please help.

I cannot emphasize enough that this is VERY important since this is directly influencing the stability/reliability of out product and it is not acceptable that the RSZ gets into a state where it require a reboot of the camera!

Since my last post I realized that the IPNC already have some provision for changing the Dynamiic BW settings and the DMM priority setting using the ./sys/bin/sys_pri.out tool.

I have tried to modify the BW and priorities according to the document you sent but I still see the RSZ stall if I run a memory BW intensive application on the ARM.

I use the following settings where I try to give the A8 lowest DMM priority (but I might be doing something wrong):

Basically I have a maximum load on the ISS bus of 33 fps @ 2560x2048.

./bin/sys_pri.out --L3-bw-reg-set HDVICP0 1 2 150 2500
./bin/sys_pri.out --L3-bw-reg-set ISS 3 3 150 2500

./bin/sys_pri.out --dmm-pri-set ISS 0
./bin/sys_pri.out --dmm-pri-set HDVICP0 2
./bin/sys_pri.out --dmm-pri-set DUCATI 1
./bin/sys_pri.out --dmm-pri-set HDVICP1 3
./bin/sys_pri.out --dmm-pri-set A8 4

Please take a look and let me know if you see anything obvious.

PS:

I also implemented a detection of the RSZ stall condition in the ISS capture driver code by comparing the number of MSP_ISS_DRV_RSZ_END_ISR interrupts against the number of MSP_ISS_DRV_ISIF_VD_ISR interrupts. So I just need a "simple" way to reset the RSZ when this happens in order to make the camera able to continue automatically.

Thanks,

Karsten