Linux/AM5718: cal driver fps speed is very slow

Nikunj Patel

Part Number: AM5718

Tool/software: Linux

Hello Everyone,

We have designed AM5718 based custom board with AR1335 Camera Sensor.

Now we are capturing the frames using the below command but the FPS Speed is very low and thus effect the our final application goal.

root@am57xx-evm:/boot# yavta -c7 -fSGRBG8 -Fvout_4208x3120_yuyv.yuv -s4208x3120 /dev/video0
Device /dev/video0 opened.
Device `cal' on `platform:cal-000' is a video output (without mplanes) device.
Video format set: SGRBG8 (47425247) 4208x3120 (stride 4208) field any buffer size 13128960
Video format: SGRBG8 (47425247) 4208x3120 (stride 4208) field any buffer size 13128960
8 buffers requested.
length: 13128960 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xb61d6000.
length: 13128960 offset: 13131776 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xb5550000.
length: 13128960 offset: 26263552 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xb48ca000.
length: 13128960 offset: 39395328 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xb3c44000.
length: 13128960 offset: 52527104 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xb2fbe000.
length: 13128960 offset: 65658880 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xb2338000.
length: 13128960 offset: 78790656 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xb16b2000.
length: 13128960 offset: 91922432 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xb0a2c000.
0 (0) [-] any 0 13128960 B 243.515647 243.515676 14.526 fps ts mono/EoF
1 (1) [-] any 1 13128960 B 243.552396 243.995571 27.212 fps ts mono/EoF
2 (2) [-] any 2 13128960 B 243.589143 244.426753 27.213 fps ts mono/EoF
3 (3) [-] any 3 13128960 B 243.625892 244.889885 27.212 fps ts mono/EoF
4 (4) [-] any 4 13128960 B 243.662639 245.325153 27.213 fps ts mono/EoF
5 (5) [-] any 5 13128960 B 243.699386 245.761801 27.213 fps ts mono/EoF
6 (6) [-] any 6 13128960 B 243.736134 246.213149 27.212 fps ts mono/EoF
Captured 7 frames in 2.766344 seconds (2.530415 fps, 33221723.502878 B/s).
8 buffers released.

We performed different test cases but we got the fps speed is 2.5 to 2.7 which is very slow for our end application.

Please provide us the support to increase the FPS(frame per second) speed.

Thanks & Best Regards;

Nikunj Patel

over 6 years ago

0 Margarita Gashova over 6 years ago

TI__Guru* 80190 points

Hello,

I guess you have modified the cal driver since the max resolution is 1920x1200 limited by the software right?
I would recommend you to check this thread:
e2e.ti.com/.../743178

Hope this helps.

BR
Margarita

0 Nikunj Patel over 6 years ago in reply to Margarita Gashova

Expert 1870 points

Hi Margarita,

We checked the above post and as per the reply by Manisha on "Nov 10, 2018 7:30 AM". Note that CAL driver will need some modification to remove the software limitations on the captured resolution so we just change the below two lines in cal driver.

#define MAX_WIDTH 4208

#define MAX_HEIGHT 3120

instead of

#define MAX_WIDTH 1920

#define MAX_HEIGHT 1200

Also we didn't found any help from your suggested link.

Note: our Pixel-rate is 495000000.

Thanks & Best Regards;

Nikunj Patel

0 Margarita Gashova over 6 years ago in reply to Nikunj Patel

TI__Guru* 80190 points

Hello Nikunj Patel,

My point was to use the formula to recalculate the FPS depending of your resolution and format.
This is 600*1000*1000/3840/2160/2 = 36.16 fps for resolution 3840x2160 YUYV(2 bytes per pixel) .
But in your case the resolution is 4208x3120 in SGRBG8 format (www.linuxtv.org/.../V4L2-PIX-FMT-SGRBG8.html)
Check the ~fps in your case.

BR
Margarita

0 Nikunj Patel over 6 years ago in reply to Margarita Gashova

Expert 1870 points

Hi Margarita;
The above link does not open.
As per my calculation as below
600*1000*1000/4208/3120/1=45.7 fps

But still I am confusion about the 600 MHz. Because my pixel rate is 495000000.

So please tell if am I wrong?

Thanks

0 Nikunj Patel over 6 years ago in reply to Nikunj Patel

Expert 1870 points

Gentle Reminder

Thanks & Best Regards;
Nikunj Patel

0 Margarita Gashova over 6 years ago in reply to Nikunj Patel

TI__Guru* 80190 points

Hello,

Sorry the right link is www.linuxtv.org/.../V4L2-PIX-FMT-SGRBG8.html

600 Mbytes/s is the maximum capture rate on CSI-2 interface.
I would recommend you to check the cal guide for more information:
processors.wiki.ti.com/.../Linux_Core_CAL_User's_Guide and ti-cal.txt file in the linux folder.

BR
Margarita

0 Nikunj Patel over 6 years ago in reply to Margarita Gashova

Expert 1870 points

Hi Margarita;

We have checked your link but there was the information for the "SGRBG8 ('GRBG')" format.

We already followed "processors.wiki.ti.com/.../Linux_Core_CAL_User's_Guide" link and as per this we have added below node in dtsi files.

&cal{
   #address-cells = <1>;
   #size-cells = <0>;
   status = "okay";

   ports{
       csi2_0: port@0 {
           reg = <0>;
           status = "okay";
           csi2_phy0: endpoint@0 {
               slave-mode;
               remote-endpoint = <&csi2_cam0>;
               };
           };
   };
};

&i2c5 {
   status = "okay";
   clock-frequency = <400000>;

    ar1335@24 {
       compatible = "onsemi,AR1335";
       reg = <0x36>;
       mux-gpios = <&gpio7 1 GPIO_ACTIVE_HIGH>;/* CSI2_CAM0_PWR_DWN */
       port {
           #address-cells = <1>;
           #size-cells = <0>;

           csi2_cam0: endpoint@0 {
               reg = <0>;
               clock-lanes = <0>;
               data-lanes = <1 2 3 4>;
               remote-endpoint = <&csi2_0>;
           };
       };
   };

};

@Margarita: the above links are not enough to solve our issue so please provide us the more support for that.

Thanks & Best Regards;

Nikunj Patel

0 Margarita Gashova over 6 years ago

TI__Guru* 80190 points

Hello,

I am looking into this.
Please could you run yavta like this:
yavta -c7 -fSGRBG8 -F/dev/null -s4208x3120 /dev/video0
and check is there a difference.

BR
Margarita

0 Nikunj Patel over 6 years ago in reply to Margarita Gashova

Expert 1870 points

Hi Margarita,

Thanks for your quick reply.

Please find the below test result

1) AR1335 sensor (Video0)

root@am57xx-evm:/mnt# yavta -c7 -fSGRBG8 -F/dev/null -s4208x3120 /dev/video0

Device /dev/video0 opened.

Device `cal' on `platform:cal-000' is a video output (without mplanes) device.

Video format set: SGRBG8 (47425247) 4208x3120 (stride 4208) field any buffer size 13128960

Video format: SGRBG8 (47425247) 4208x3120 (stride 4208) field any buffer size 13128960

8 buffers requested.

length: 13128960 offset: 0 timestamp type/source: mono/EoF

Buffer 0/0 mapped at address 0xb61e4000.

length: 13128960 offset: 13131776 timestamp type/source: mono/EoF

Buffer 1/0 mapped at address 0xb555e000.

length: 13128960 offset: 26263552 timestamp type/source: mono/EoF

Buffer 2/0 mapped at address 0xb48d8000.

length: 13128960 offset: 39395328 timestamp type/source: mono/EoF

Buffer 3/0 mapped at address 0xb3c52000.

length: 13128960 offset: 52527104 timestamp type/source: mono/EoF

Buffer 4/0 mapped at address 0xb2fcc000.

length: 13128960 offset: 65658880 timestamp type/source: mono/EoF

Buffer 5/0 mapped at address 0xb2346000.

length: 13128960 offset: 78790656 timestamp type/source: mono/EoF

Buffer 6/0 mapped at address 0xb16c0000.

length: 13128960 offset: 91922432 timestamp type/source: mono/EoF

Buffer 7/0 mapped at address 0xb0a3a000.

0 (0) [-] any 0 13128960 B 2248.307057 2248.307073 14.520 fps ts mono/EoF

1 (1) [-] any 1 13128960 B 2248.343805 2248.343819 27.212 fps ts mono/EoF

2 (2) [-] any 2 13128960 B 2248.380552 2248.380565 27.213 fps ts mono/EoF

3 (3) [-] any 3 13128960 B 2248.417302 2248.417319 27.211 fps ts mono/EoF

4 (4) [-] any 4 13128960 B 2248.454046 2248.454059 27.215 fps ts mono/EoF

5 (5) [-] any 5 13128960 B 2248.490793 2248.490807 27.213 fps ts mono/EoF

6 (6) [-] any 6 13128960 B 2248.527541 2248.527554 27.212 fps ts mono/EoF

Captured 7 frames in 0.289369 seconds (24.190550 fps, 317596763.133005 B/s).

8 buffers released.

2) Virtual vivid Driver(video1)

root@am57xx-evm:/mnt# modprobe vivid

root@am57xx-evm:/mnt# yavta -c7 -fSGRBG8 -F/dev/null -s4208x3120 /dev/video1

Device /dev/video1 opened.

Device `vivid' on `platform:vivid-000' is a video output (without mplanes) device.

Video format set: SGRBG8 (47425247) 4208x3120 (stride 4208) field none buffer size 13128960

Video format: SGRBG8 (47425247) 4208x3120 (stride 4208) field none buffer size 13128960

8 buffers requested.

length: 13128960 offset: 0 timestamp type/source: mono/EoF

Buffer 0/0 mapped at address 0xb61ef000.

length: 13128960 offset: 13131776 timestamp type/source: mono/EoF

Buffer 1/0 mapped at address 0xb5569000.

length: 13128960 offset: 26263552 timestamp type/source: mono/EoF

Buffer 2/0 mapped at address 0xb48e3000.

length: 13128960 offset: 39395328 timestamp type/source: mono/EoF

Buffer 3/0 mapped at address 0xb3c5d000.

length: 13128960 offset: 52527104 timestamp type/source: mono/EoF

Buffer 4/0 mapped at address 0xb2fd7000.

length: 13128960 offset: 65658880 timestamp type/source: mono/EoF

Buffer 5/0 mapped at address 0xb2351000.

length: 13128960 offset: 78790656 timestamp type/source: mono/EoF

Buffer 6/0 mapped at address 0xb16cb000.

length: 13128960 offset: 91922432 timestamp type/source: mono/EoF

Buffer 7/0 mapped at address 0xb0a45000.

0 (0) [-] none 0 13128960 B 2270.993814 2270.994154 63.739 fps ts mono/EoF

1 (1) [-] none 1 13128960 B 2271.055654 2271.058584 16.171 fps ts mono/EoF

2 (2) [-] none 2 13128960 B 2271.115679 2271.118611 16.660 fps ts mono/EoF

3 (3) [-] none 3 13128960 B 2271.185695 2271.188625 14.282 fps ts mono/EoF

4 (4) [-] none 4 13128960 B 2271.255610 2271.258532 14.303 fps ts mono/EoF

5 (5) [-] none 5 13128960 B 2271.315717 2271.318673 16.637 fps ts mono/EoF

6 (6) [-] none 6 13128960 B 2271.385699 2271.388629 14.289 fps ts mono/EoF

Captured 7 frames in 0.410503 seconds (17.052230 fps, 223878051.178733 B/s).

8 buffers released.

As per above checking we got the fps speed "24.190550" but we have done same test on virtual driver as per above.
We found that the first frame capture fps is "14.520" on our sensor whenever "63.739" fps on virtual driver. so can you help us for why this happens because our End Application framework only captures one frame at a time and stored then again capture and stored.

Also one more thing that how above captured data will be validated for correct or incorrect?

Thanks & Best Regards;

Nikunj Patel

0 Nikunj Patel over 6 years ago in reply to Nikunj Patel

Expert 1870 points

gentle reminder,

Thanks & Best Regards;
Nikunj Patel

0 Margarita Gashova over 6 years ago in reply to Nikunj Patel

TI__Guru* 80190 points

Hello,

The fps 2.5 that you are observing with yavta is due the file write.
That is why you are seeing fps ~ 27 when there is no save in file.
Regarding the fps of 14 that you are observing, I am seeing the same when I capture from vip via yavta.
Let me consult with the expert for the possible reason.
I will get back to you when I have something.

BR
Margarita

0 Nikunj Patel over 6 years ago in reply to Margarita Gashova

Expert 1870 points

HI Margarita,

Very Very thanks for the support.

Best Regards;
Nikunj Patel

0 Manisha Agrawal over 6 years ago in reply to Nikunj Patel

TI__Mastermind 22946 points

Hi Nikunj,

You should ignore the first fps print from Yavta.

Yavta (git://git.ideasonboard.org/yavta.git) is a fairly simple test application and the code is relatively easy to follow as it is a single source C file.

The way yavta reports the per frame fps value is by calculating the inter-frame delay. Yavta starts the timer right after the STREAMON is issued. Now when the STREAMON is issued, the capture engine is configured and started and the STREAMON message is also forwarded to the camera driver which in most cases will then initialize the sensor itself by programming the sensor register usually through a "slow" link like I2C. All of this creates latency/delay for the first frame, which is why it shows the lower fps value for first frame and because of this you should ignore first fps print.

You don't see this for virtual driver, because it is all software driven and designed to generate frames at 25/30 fps.

Regards,
Manisha

0 Nikunj Patel over 6 years ago in reply to Manisha Agrawal

Expert 1870 points

Hi Manisha,

Thanks for your reply.
We are doing same test with the vivid driver in our framework where we got 4-5 fps but when we used the "cal[AR1335]" sensor driver so we got only 1 fps so i think something is missed by us. In our framework we are doing only one request at a time as
"yavta -c1 -fSGRBG8 -s4208x3120 /dev/video0" command. so is there anyway to increase the first frame fps rate because all the other things are same.

Thanks & Best Regards
Nikunj Patel

0 Nikunj Patel over 6 years ago in reply to Nikunj Patel

Expert 1870 points

Hi Manisha,
is there any application which is better then yavta for meausring the fps speed?

Thanks & Best Regards;
Nikunj

0 Nikunj Patel over 6 years ago in reply to Manisha Agrawal

Expert 1870 points

Hello everyone,

Gentle reminder

can we read the sensor data line by line?

Thanks & Best Regards,
Nikunj Patel

0 Margarita Gashova over 6 years ago in reply to Nikunj Patel

TI__Guru* 80190 points

Hello Nikunj Patel,

Let as check we will get back to you when we have an answer.
Thank you.

BR
Margarita

0 Nikunj Patel over 6 years ago in reply to Margarita Gashova

Expert 1870 points

Thanks Margarita

BR
Nikunj Patel

0 Manisha Agrawal over 6 years ago in reply to Nikunj Patel

TI__Mastermind 22946 points

Hi Nikunj,

With respect to fps measurement, you need to understand how fps are measured. It is the timing between two captured frames once both the sensor and VIP IP are initialized. V4L2 API STREAMON wraps under-hood the handshake and initialization part. When the STREAMON is issued, the capture engine is configured and started and the STREAMON message is also forwarded to the camera driver which in most cases will then initialize the sensor itself by programming the sensor register usually through a "slow" link like I2C. Any timing overhead reported on first frame fps, it comes from this capture engine and sensor driver initialization. I am not aware of any application that reports this delay and incorporate this in reporting the 1st frame capture. I suggest you add the time stamp of driver initialization in your sensor driver code and modify Yavta code to subtract that from the 1st captured frame timestamp. You can use alternate means as well, but this is something that you need to play. This is open source and simple application to play with.

Regarding, sensor data read line by line, this is not supported.

Regards,
Manisha

Processors

Processors forum

Linux/AM5718: cal driver fps speed is very slow