SK-AM62A-LP: Configuring a new CSI-2 Image Sensor to use ISP VPAC Functionality

Hi Steve,

I have assigned your query to our expert. He is currently out of office. Please expect a response during mid week when he returns.

In the meantime, when you login as root into the EVM, Does it show the details of connected camera?

Like this with my IMX219:

Best Regards,

Suren

Hello Suren,

Thank you for the update and support.

It shows that when I log in as root with the IMX219 camera connected (and configured to use the IMX219 overlay) but nothing is printed when I have it configured to use the OV5647 camera. Additionally, I can successfully stream video using the IMX219 camera as well.

Best Regards,

Steve

Hello Steve,

Thanks for providing the detailed information. I noticed a typo in your "ov5647_properties.txt":

SENSOR_ID OV5647

It should be just "5647" to match what you have in tiovx_sensor_module.c:

SENSOR_ID 5647

Please correct this and give it a try.

Also, you may want to just use Yavta to capture the raw image to verify that the sensor driver is working fine within the V4L2 framework. Please refer here for examples of using Yavta. Once that is verified, you can move to GStreamer and include ISP to the pipeline.

Regards,

Jianzhong

Hello Jianzhong,

Thank you for your assitance. My apologies, it looks like I accidentally did the typo when I was entering the information into my post actually. Here are the actual/corrected contents of the ov5647_properties.txt file (copy and pasted this time):

WIDTH 1920
HEIGHT 1280
# BIT_DEPTH may be 12, 10, or 8 for linear mode
BIT_DEPTH 10
# 0=RGGB; 1=GRBG; 2=GBRG; 3=BGGR
BAYER_PATTERN 3
SENSOR_ID 5647
PRJ_DIR ../../../../imaging/sensor_drv/src/ov5647
SENSOR_NAME ov5647
SENSOR_DCC_NAME SENSOR_ONSEMI_OV5647

I believe I have seen an error message stating that the IDs don't match when running the gst-launch command if the SENSOR_IDs don't match which wasn't in the output that I posted. I additionally regenerated the files using the above config and they were identical to the ones I used in the original post, when checked with a binary compare tool.

Here is the output from using yavta:

root@am62axx-evm:/opt/imaging/ov5647/captures# yavta -c -Fcapture -s 1920x1080 -f SBGGR10 /dev/video2
Device /dev/video2 opened.
Device `j721e-csi2rx' on `platform:30102000.ticsi2rx' is a video output (without mplanes) device.
Video format set: SBGGR10 (30314742) 1920x1080 (stride 3840) field none buffer size 4147200
Video format: SBGGR10 (30314742) 1920x1080 (stride 3840) field none buffer size 4147200
8 buffers requested.
length: 4147200 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffff8def7000.
length: 4147200 offset: 4149248 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xffff8db02000.
length: 4147200 offset: 8298496 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffff8d70d000.
length: 4147200 offset: 12447744 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffff8d318000.
length: 4147200 offset: 16596992 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffff8cf23000.
length: 4147200 offset: 20746240 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffff8cb2e000.
length: 4147200 offset: 24895488 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffff8c739000.
length: 4147200 offset: 29044736 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffff8c344000.
0 (0) [-] any 0 4147200 B 1473.789585 1473.789639 -27777.778 fps ts mono/EoF
1 (1) [-] any 1 4147200 B 1473.872211 1473.872236 12.103 fps ts mono/EoF
2 (2) [-] any 2 4147200 B 1473.902451 1473.924595 33.069 fps ts mono/EoF
3 (3) [-] any 3 4147200 B 1473.932686 1473.976789 33.074 fps ts mono/EoF
4 (4) [-] any 4 4147200 B 1473.962927 1474.029067 33.068 fps ts mono/EoF
5 (5) [-] any 5 4147200 B 1473.993175 1474.081058 33.060 fps ts mono/EoF
6 (6) [-] any 6 4147200 B 1474.023409 1474.133922 33.075 fps ts mono/EoF
7 (7) [-] any 7 4147200 B 1474.053652 1474.186016 33.066 fps ts mono/EoF
8 (0) [-] any 8 4147200 B 1474.083890 1474.238562 33.071 fps ts mono/EoF
9 (1) [-] any 9 4147200 B 1474.114130 1474.290602 33.069 fps ts mono/EoF
10 (2) [-] any 10 4147200 B 1474.144373 1474.344759 33.066 fps ts mono/EoF
11 (3) [-] any 11 4147200 B 1474.174609 1474.398590 33.073 fps ts mono/EoF
12 (4) [-] any 12 4147200 B 1474.204861 1474.452699 33.056 fps ts mono/EoF
13 (5) [-] any 13 4147200 B 1474.235093 1474.504754 33.078 fps ts mono/EoF
14 (6) [-] any 14 4147200 B 1474.265338 1474.558820 33.063 fps ts mono/EoF
15 (7) [-] any 15 4147200 B 1474.295580 1474.612683 33.067 fps ts mono/EoF
16 (0) [-] any 16 4147200 B 1474.325812 1474.666738 33.078 fps ts mono/EoF
17 (1) [-] any 17 4147200 B 1474.386293 1474.718748 16.534 fps ts mono/EoF
18 (2) [-] any 18 4147200 B 1474.446776 1474.772856 16.534 fps ts mono/EoF
19 (3) [-] any 19 4147200 B 1474.507262 1474.826901 16.533 fps ts mono/EoF
20 (4) [-] any 20 4147200 B 1474.537498 1474.880949 33.073 fps ts mono/EoF
21 (5) [-] any 21 4147200 B 1474.597980 1474.934637 16.534 fps ts mono/EoF
22 (6) [-] any 22 4147200 B 1474.658458 1474.986705 16.535 fps ts mono/EoF
23 (7) [-] any 23 4147200 B 1474.718946 1475.041727 16.532 fps ts mono/EoF
24 (0) [-] any 24 4147200 B 1474.749182 1475.095990 33.073 fps ts mono/EoF
25 (1) [-] any 25 4147200 B 1474.809663 1475.149772 16.534 fps ts mono/EoF
26 (2) [-] any 26 4147200 B 1474.870135 1475.202104 16.537 fps ts mono/EoF
27 (3) [-] any 27 4147200 B 1474.930618 1475.255926 16.534 fps ts mono/EoF
28 (4) [-] any 28 4147200 B 1474.991098 1475.310000 16.534 fps ts mono/EoF
29 (5) [-] any 29 4147200 B 1475.021342 1475.363808 33.064 fps ts mono/EoF
30 (6) [-] any 30 4147200 B 1475.081827 1475.417879 16.533 fps ts mono/EoF
31 (7) [-] any 31 4147200 B 1475.142303 1475.470422 16.535 fps ts mono/EoF
32 (0) [-] any 32 4147200 B 1475.202781 1475.524587 16.535 fps ts mono/EoF
33 (1) [-] any 33 4147200 B 1475.233023 1475.578266 33.067 fps ts mono/EoF
34 (2) [-] any 34 4147200 B 1475.293504 1475.632309 16.534 fps ts mono/EoF
35 (3) [-] any 35 4147200 B 1475.353987 1475.684309 16.534 fps ts mono/EoF
36 (4) [-] any 36 4147200 B 1475.414475 1475.738426 16.532 fps ts mono/EoF
37 (5) [-] any 37 4147200 B 1475.474947 1475.792254 16.537 fps ts mono/EoF
38 (6) [-] any 38 4147200 B 1475.505185 1475.846820 33.071 fps ts mono/EoF
39 (7) [-] any 39 4147200 B 1475.565666 1475.900739 16.534 fps ts mono/EoF
40 (0) [-] any 40 4147200 B 1475.626152 1475.952703 16.533 fps ts mono/EoF
41 (1) [-] any 41 4147200 B 1475.686630 1476.006395 16.535 fps ts mono/EoF
42 (2) [-] any 42 4147200 B 1475.716877 1476.060580 33.061 fps ts mono/EoF
43 (3) [-] any 43 4147200 B 1475.777353 1476.114459 16.535 fps ts mono/EoF
44 (4) [-] any 44 4147200 B 1475.837834 1476.166651 16.534 fps ts mono/EoF
45 (5) [-] any 45 4147200 B 1475.898311 1476.220659 16.535 fps ts mono/EoF
^C

I'm not sure of how to check this file currently though. The 7yuv and PixelViewer websites do not make me feel comfortable from a security perspective and I really would like to not install 3rd party software, when possible.

I've alternatively done as outlined in the SPRAD86 AM6xA ISP Tuning Guide and captured some raw images using the command:

root@am62axx-evm:/opt/imaging/ov5647/captures# gst-launch-1.0 -v v4l2src num-buffers=5 device=/dev/video2 io-mode=dmabuf ! video/x-bayer, width=1920, height=1080, framerate=30/1, format=bggr10 ! multifilesink location="ov5647-image-%d.raw"
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)bggr10, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)bggr10, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstMultiFileSink:multifilesink0.GstPad:sink: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)bggr10, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)bggr10, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.325626125
Setting pipeline to NULL ...
Freeing pipeline ...
root@am62axx-evm:/opt/imaging/ov5647/captures# ls
ov5647-image-0.raw  ov5647-image-1.raw  ov5647-image-2.raw  ov5647-image-3.raw  ov5647-image-4.raw

I was able to look at the last couple raw images output from this command using Vooya (which I had previously installed) and it showed a black and white image captured from the image sensor.

-Steve

Hi Steve,

Steve T said:

it showed a black and white image captured from the image sensor.

Yes, it should be black and white. Does the image look correct except the colors?

Thanks,

Jianzhong

Yes, the image looks correct.

Ok. That's good news. It means the GStreamer pipeline from camera to DDR without using the ISP is working as expected. Can you try to add ISP to the working GStreamer pipeline?

Also, I noticed that your original GStreamer pipeline didn't specify the video format for display. Can you follow the example given in the ISP tuning guide?

Regards,

Jianzhong

No change was apparent to me by adding in the video format for the display but I will keep it in the command from this point forward.

root@am62axx-evm:/opt/edgeai-gst-apps# GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element+pipeline)" gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video2 ! video/x-bayer, width=1920, height=1080, format=bggr10, framerate=30/1 ! tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name=SENSOR_ONSEMI_OV5647 dcc-isp-file=/opt/imaging/ov5647/dcc_viss_10b.bin sink_0::dcc-2a-file=/opt/imaging/ov5647/dcc_2a_10b.bin format-msb=9 ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! kmssink driver-name=tidss sync=false
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   290.314191 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   290.314394 s:  VX_ZONE_INIT:Enabled
   290.314435 s:  VX_ZONE_ERROR:Enabled
   290.314466 s:  VX_ZONE_WARNING:Enabled
   290.315178 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   290.315687 s:  VX_ZONE_INIT:[tivxHostInitLocal:93] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:10.521301410
Setting pipeline to NULL ...
^C

Also, the output from the gst-tracers still only had the same one item listed.

Best Regards,

Steve

How about writing to files instead of displaying? 

gst-launch-1.0 v4l2src num-buffers=$1 io-mode=dmabuf device=/dev/video2 ! video/x-bayer, width=1920, height=1080, format=bggr10 ! \
tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name=SENSOR_ONSEMI_OV5647 dcc-isp-file=/opt/imaging/ov5647/dcc_viss_10b.bin \
sink_0::dcc-2a-file=/opt/imaging/ov5647/dcc_2a_10b.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1080 ! jpegenc ! multifilesink location="ov5647-image-%d.jpg"

This also doesn't ever complete. I believe there is a typo in the command of num-buffers=$1. Here are outputs from testing:

root@am62axx-evm:/opt/edgeai-gst-apps# media-ctl --set-v4l2 "'ov5647 4-0036':0[fmt:SBGGR10_1X10/1920x1080 field:none]"
root@am62axx-evm:/opt/edgeai-gst-apps# cd /opt/imaging/ov5647/captures/
root@am62axx-evm:/opt/imaging/ov5647/captures# ls
capture  ov5647-image-0.raw  ov5647-image-1.raw  ov5647-image-2.raw  ov5647-image-3.raw  ov5647-image-4.raw
root@am62axx-evm:/opt/imaging/ov5647/captures# gst-launch-1.0 v4l2src num-buffers=$1 io-mode=dmabuf device=/dev/video2 ! video/x-bayer, width=1920, height=1080, format=bggr10 ! \
> tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name=SENSOR_ONSEMI_OV5647 dcc-isp-file=/opt/imaging/ov5647/dcc_viss_10b.bin \
> sink_0::dcc-2a-file=/opt/imaging/ov5647/dcc_2a_10b.bin format-msb=9 ! \
> video/x-raw, format=NV12, width=1920, height=1080 ! jpegenc ! multifilesink location="ov5647-image-%d.jpg"
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   174.468273 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   174.474847 s:  VX_ZONE_INIT:Enabled
   174.474893 s:  VX_ZONE_ERROR:Enabled
   174.474909 s:  VX_ZONE_WARNING:Enabled
   174.476075 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   174.476567 s:  VX_ZONE_INIT:[tivxHostInitLocal:93] Initialization Done for HOST !!!
WARNING: erroneous pipeline: could not set property "num-buffers" in element "v4l2src0" to "io-mode=dmabuf"
root@am62axx-evm:/opt/imaging/ov5647/captures# gst-launch-1.0 v4l2src num-buffers=5 io-mode=dmabuf device=/dev/video2 ! video/x-bayer, width=1920, height=1080, format=bggr10 ! tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name=SENSOR_ONSEMI_OV5647 dcc-isp-file=/opt/imaging/ov5647/dcc_viss_10b.bin sink_0::dcc-2a-file=/opt/imaging/ov5647/dcc_2a_10b.bin format-msb=9 ! video/x-raw, format=NV12, width=1920, height=1080 ! jpegenc ! multifilesink location="ov5647-image-%d.jpg"
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   248.038054 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   248.038174 s:  VX_ZONE_INIT:Enabled
   248.038192 s:  VX_ZONE_ERROR:Enabled
   248.038204 s:  VX_ZONE_WARNING:Enabled
   248.038920 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   248.039849 s:  VX_ZONE_INIT:[tivxHostInitLocal:93] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:17.373220000
Setting pipeline to NULL ...
^C
root@am62axx-evm:/opt/imaging/ov5647/captures# ls
capture  ov5647-image-0.raw  ov5647-image-1.raw  ov5647-image-2.raw  ov5647-image-3.raw  ov5647-image-4.raw

Steve,

If we are encoding the files into jpeg, then why are the files dumped as raw files?

Yes num-buffers=1 is correct may be it was a typo as $1

Are the jpeg files not capturing the camera data from num-buffers=5??

Regards,

Suren

Suren,

Sorry if it was confusing but you can see that those files were previously there in the initial "ls" command on the directory. Those .raw files are from following the SPRAD86 AM6xA ISP Tuning Guide method to check the camera functionality as previously discussed in the thread. That was how I was able to verify that the camera was outputting a black and white image.

Correct, the jpeg files are not capturing the camera data when using num-buffers=5. The command hangs.

Best Regards,

Steve

Steve,

Is the problem only with the newly tuned senor(OV5647) or you are observing the problem with IMX219 as well?

After tuning, are you able to display camera capture using kmssink on your display/hdmi?

Also upon boot, is your camera enumerated and displayed on the console?

I am not seeing any issues with the current gstreamer command on my setup with IMX219:

GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \
gst-launch-1.0 v4l2src device=/dev/video2 io-mode=dmabuf-import num-buffers=5 ! \
video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb10 ! \
tiovxisp sink_0::device=/dev/v4l-subdev2 \
sensor-name="SENSOR_SONY_IMX219_RPI" \
dcc-isp-file=/opt/imaging/imx219/dcc_viss_10b.bin \
sink_0::dcc-2a-file=/opt/imaging/imx219/dcc_2a_10b.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
jpegenc ! multifilesink location=/home/img_%06d.jpg

Best Regards,

Suren

Suren,

"Is the problem only with the newly tuned senor(OV5647) or you are observing the problem with IMX219 as well?"

I'm only seeing the issue with the OV5647. As I stated previously, "I can successfully stream video using the IMX219 camera". I would however like to use other image sensors than the handful of ones provided in the SDK. Is that not fully supported?

"After tuning, are you able to display camera capture using kmssink on your display/hdmi?"

I'm not totally sure what you mean by tuning here so if you could elaborate more that could be helpful to me. So far, all I've done is enable the native driver module in the Linux kernel, create a dtb file for the sensor configuration, update the kernel, copy over the dtb while updating the uImage to use it, and provide the built Imaging ISP files. I have yet to see the OV5647 a captured image on the my HDMI display since the gstreamer command always hangs. I haven't done anything to adjust the auto white balance, etc. setting yet. I have successfully seen an output window on my HDMI display using both a Gstreamer test pattern and the IMX219 so I know it is capable displaying there.

"Also upon boot, is your camera enumerated and displayed on the console?"

As I stated when you previously asked, "It shows that when I log in as root with the IMX219 camera connected (and configured to use the IMX219 overlay) but nothing is printed when I have it configured to use the OV5647 camera." I don't think it would because the setup_camera.sh script wouldn't catch it, as far as I can tell.

I feel like you have asked questions that I have already answered. Please do your due diligence and read through the previous posts thoroughly so we can make timely and forward progress.

Best Regards,

Steve

Hi Steve

Steve T said:

I believe there is a typo in the command of num-buffers=$1

Yes, that is a typo. The command was copied from a script which gets num-buffers from the argument. I apologize for that.

Have you added a function to configure the auto exposure parameters for the sensor?

You'll also need to add the calling of this function similarly to IMX219:

if (g_strcmp0 (self->sensor_name, "SENSOR_SONY_IMX390_UB953_D3") == 0) {
      get_imx390_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
    } else if (g_strcmp0 (self->sensor_name, "SENSOR_OV2312_UB953_LI") == 0) {
      get_ov2312_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
    } else {
      get_imx219_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
    }

Regards,

Jianzhong

Jianzhong,

I have added the following lines of code into /opt/edgeai-gst-plugins/ext/tiovx/gsttiovxisp.c:

....
static int32_t get_imx219_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms);

static int32_t get_ov5647_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms);

static int32_t get_imx390_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms);
....

....
    if (g_strcmp0 (self->sensor_name, "SENSOR_SONY_IMX390_UB953_D3") == 0) {
      get_imx390_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
    } else if (g_strcmp0 (self->sensor_name, "SENSOR_OV2312_UB953_LI") == 0) {
      get_ov2312_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
    } else if (g_strcmp0 (self->sensor_name, "SENSOR_ONSEMI_OV5647") == 0) {
      get_ov5647_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
    } else {
      get_imx219_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
    }
....

....
static int32_t
get_ov5647_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms)
{
  int32_t status = -1;
  uint8_t count = 0;

  g_return_val_if_fail (p_ae_dynPrms, status);

  p_ae_dynPrms->targetBrightnessRange.min = 40;
  p_ae_dynPrms->targetBrightnessRange.max = 50;
  p_ae_dynPrms->targetBrightness = 45;
  p_ae_dynPrms->threshold = 1;
  p_ae_dynPrms->enableBlc = 1;
  p_ae_dynPrms->exposureTimeStepSize = 1;

  p_ae_dynPrms->exposureTimeRange[count].min = 100;
  p_ae_dynPrms->exposureTimeRange[count].max = 33333;
  p_ae_dynPrms->analogGainRange[count].min = 1024;
  p_ae_dynPrms->analogGainRange[count].max = 8192;
  p_ae_dynPrms->digitalGainRange[count].min = 256;
  p_ae_dynPrms->digitalGainRange[count].max = 256;
  count++;

  p_ae_dynPrms->numAeDynParams = count;
  status = 0;
  return status;
}
....

I'm not sure how the parameters in the function pertain to the ov5647 sensor so not really sure what they should be changed to but I was hoping to at least see an error. Instead I have again observed no camera video window or difference when running the commands:

root@am62axx-evm:/opt/edgeai-gst-apps# media-ctl --set-v4l2 "'ov5647 4-0036':0[fmt:SBGGR10_1X10/1920x1080 field:none]"
root@am62axx-evm:/opt/edgeai-gst-apps# GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element+pipeline)" gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video2 ! video/x-bayer, width=1920, height=1080, format=bggr10, framerate=30/1 ! tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name=SENSOR_ONSEMI_OV5647 dcc-isp-file=/opt/imaging/ov5647/dcc_viss_10b.bin sink_0::dcc-2a-file=/opt/imaging/ov5647/dcc_2a_10b.bin format-msb=9 ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! kmssink driver-name=tidss sync=false
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   689.892970 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   689.899754 s:  VX_ZONE_INIT:Enabled
   689.899802 s:  VX_ZONE_ERROR:Enabled
   689.899816 s:  VX_ZONE_WARNING:Enabled
   689.900748 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   689.901309 s:  VX_ZONE_INIT:[tivxHostInitLocal:93] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:11.535338580
Setting pipeline to NULL ...
^C

Also, I still only see capsfilter0 in the output of the gst_tracers script.

Here is the list of V4L2 parameters for the camera, in case that helps:

root@am62axx-evm:/opt/edgeai-gst-apps# v4l2-ctl -d /dev/v4l-subdev2 -list
VIDIOC_S_INPUT: failed: Inappropriate ioctl for device

User Controls

        white_balance_automatic 0x0098090c (bool)   : default=0 value=0
                       exposure 0x00980911 (int)    : min=4 max=500 step=1 default=500 value=500
                 gain_automatic 0x00980912 (bool)   : default=0 value=0

Camera Controls

                  auto_exposure 0x009a0901 (menu)   : min=0 max=1 default=1 value=1 (Manual Mode)

Image Source Controls

              vertical_blanking 0x009e0901 (int)    : min=4 max=32287 step=1 default=24 value=24
            horizontal_blanking 0x009e0902 (int)    : min=1212 max=1212 step=1 default=1212 value=1212 flags=read-only
                  analogue_gain 0x009e0903 (int)    : min=16 max=1023 step=1 default=32 value=32

Image Processing Controls

                     pixel_rate 0x009f0902 (int64)  : min=55000000 max=55000000 step=1 default=55000000 value=55000000 flags=read-only

Best Regards,

Steve

Hello Steve,

Most likely the problem is with the ISP profile binaries. I'm traveling now and will be back to office on Wednesday. I'll try to replicate what you did and see if I can either reproduce what you saw or make it work. Would you mind sharing where you got the camera from?

Thanks and regards,

Jianzhong

Hello Jianzhong,

Understood. Thank you for your continued support.

Here is a link to the camera manufacturer's webpage: https://www.arducam.com/product/arducam-5mp-m12-picam/

You can likely buy it off of Amazon from Uctronic's (same company linked to on the Buy Now on Arducam's webpage) store here:

https://www.amazon.com/Arducam-Camera-Raspberry-Interchangeable-M12x0-5/dp/B013JTY8WY/ref=sr_1_9?crid=1FWU9XVXESGDG&keywords=Arducam+5MP+OV5647+Camera&qid=1684859333&sprefix=arducam+5mp+ov5647+camera%2Caps%2C101&sr=8-9

I additionally bought a 22-to-15 pin cable to be able to attach it to the SK-AM62A-LP.

Best Regards,

Steve

Hi Steve,

Thanks for providing the information. I have a similar OV5647 module and I'll try with that one first.

Steve T said:

The image sensor driver is natively in the Linux kernel provided in the SDK and has been enabled as a module. The kernel, dtbo and modules were built and copied over to the SD card, following the instructions on the Academy here

Would it be possible for you to share a patch of the changes you made? I just wanted to double check on that and make sure nothing was missed.

Regards,

Jianzhong

Jianzhong,

I'm not sure of what you mean by a patch of the changes (like a git diff perhaps?) or how to accomplish that so if you could provide some detailed steps of what to do I would appreciate your assistance with providing that. I've a few other changes to the kernel though as I've been working on development for a project.

So instead I'm providing my image sensor dts file and the Makefile and the generated dtbo file from my "../<ti-sdk>/board-support/<linux-5.10.168+...>/arch/arm64/boot/dts/ti/" folder. I've also attached my bin files generated using the imaging tools.

Hopefully these are useful at least.

4251.ov5647.zip

Best Regards,

Steve

Steve,

Steve T said:

I'm not sure of what you mean by a patch of the changes (like a git diff perhaps?)

Yes, that's what I meant. What you provided should be sufficient.

Thanks and regards,

Jianzhong

Hello Steve,

Sorry for the delay. I bought the same OV5647 camera that you're using. I followed the procedure provided in the ISP tuning guide and am able to stream using the default ISP configuration.

I had the following auto exposure setting just for testing.

diff --git a/gsttiovxisp-orig.c b/gsttiovxisp.c
index 21693a5..eaed403 100644
--- a/gsttiovxisp-orig.c
+++ b/gsttiovxisp.c
@@ -624,6 +624,8 @@ static const gchar *target_id_to_target_name (gint target_id);
 
 static int32_t get_imx219_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms);
 
+static int32_t get_ov5647_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms);
+
 static int32_t get_imx390_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms);
 
 static int32_t get_ov2312_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms);
@@ -676,6 +678,7 @@ gst_tiovx_isp_class_init (GstTIOVXISPClass * klass)
           "                                   SENSOR_ONSEMI_AR0820_UB953_LI\n"
           "                                   SENSOR_ONSEMI_AR0233_UB953_MARS\n"
           "                                   SENSOR_SONY_IMX219_RPI\n"
+          "                                   SENSOR_OV5647_RPI\n"
           "                                   SENSOR_OV2312_UB953_LI",
           NULL,
           G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS |
@@ -1765,6 +1768,8 @@ gst_tiovx_isp_postprocess (GstTIOVXMiso * miso)
       get_imx390_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
     } else if (g_strcmp0 (self->sensor_name, "SENSOR_OV2312_UB953_LI") == 0) {
       get_ov2312_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
+    } else if (g_strcmp0 (self->sensor_name, "SENSOR_OV5647_RPI") == 0) {
+      get_ov5647_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
     } else {
       get_imx219_ae_dyn_params (&sink_pad->sensor_in_data.ae_dynPrms);
     }
@@ -1872,6 +1877,34 @@ get_imx219_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms)
   return status;
 }
 
+static int32_t                                                                  
+get_ov5647_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms)                
+{                                                                                  
+  int32_t status = -1;                                                      
+  uint8_t count = 0;                                                        
+                                                                                
+  g_return_val_if_fail (p_ae_dynPrms, status);                                  
+                                                                                
+  p_ae_dynPrms->targetBrightnessRange.min = 40;                                    
+  p_ae_dynPrms->targetBrightnessRange.max = 50;                                
+  p_ae_dynPrms->targetBrightness = 45;                                        
+  p_ae_dynPrms->threshold = 1;                                            
+  p_ae_dynPrms->enableBlc = 1;                                                 
+  p_ae_dynPrms->exposureTimeStepSize = 1;                                      
+                                                                            
+  p_ae_dynPrms->exposureTimeRange[count].min = 4;                         
+  p_ae_dynPrms->exposureTimeRange[count].max = 500;                       
+  p_ae_dynPrms->analogGainRange[count].min = 16;                        
+  p_ae_dynPrms->analogGainRange[count].max = 1023;                        
+  p_ae_dynPrms->digitalGainRange[count].min = 256;                             
+  p_ae_dynPrms->digitalGainRange[count].max = 256;                             
+  count++;                                               
+                                                                        
+  p_ae_dynPrms->numAeDynParams = count;                                        
+  status = 0;                                                                  
+  return status;                                                          
+}                                                                               
+
 static int32_t
 get_imx390_ae_dyn_params (IssAeDynamicParams * p_ae_dynPrms)
 {
@@ -1968,6 +2001,13 @@ gst_tiovx_isp_map_2A_values (GstTIOVXISP * self, int exposure_time,
 
     multiplier = analog_gain / 1024.0;
     *analog_gain_mapped = 256.0 - 256.0 / multiplier;
+  } else if (g_strcmp0 (self->sensor_name, "SENSOR_OV5647_RPI") == 0) {
+    /* to add setting for OV5647 */ 
+    double multiplier = 0;
+    *exposure_time_mapped = (1080 * exposure_time / 33);                    
+                                                                         
+    multiplier = analog_gain / 1024.0;                                          
+    *analog_gain_mapped = 256.0 - 256.0 / multiplier;                       
   } else if (g_strcmp0 (self->sensor_name, "SENSOR_OV2312_UB953_LI") == 0) {
     *exposure_time_mapped = (60 * 1300 * exposure_time / 1000000);
     // ms to row_time conversion - row_time(us) = 1000000/fps/height

We need to understand the format of these settings used by the sensor driver in order to derive the right values. However even these settings are not optimal, streaming should still be able to run. It's just that the image quality may not be good.

Regards,

Jianzhong

Hi Jianzhong,

Thank you for the additional support of acquiring the camera and getting it to work on your system. It's a good data point to know that it can work. Unfortunately, even after adding the the exposure time/analog gain section and matching other setting in my gsttiovxisp.c file I am still getting the same results. It isn't obvious to me what might be causing this currently. I have attached my current "gsttiovxisp.c" file and the build is saying the Project Version is 0.7.0 for GstTIOVX.

gsttiovxisp.c

How were you able to do a git diff on the "gsttiovxisp.c" file?  There isn't a git repository file/configuration natively in my system that I can tell.

You started with the "tisdk-edgeai-image-am62axx-evm.wic.xz" image, correct?

Would you be able to provide your imaging bin files? (If they are actually different using a binary compare tool such as "diff" in linux)

Any other suggestions on things to check or information I can provide to help troubleshoot?

Best Regards,

Steve

Hello Jianzhong,

Ah, I follow how you are using git now. Thank you!

I was able to successfully stream images from the camera using your bin files. This lead me to look at the config file again and now I finally see my that the HEIGHT is set to 1280 instead 1080 by mistake. I'm feel a little silly that I missed this up until now. >_<. I was able to regenerate the binary files with HEIGHT set to 1080 and use those to successfully to stream, as well. Hooray!

Here is the output now from the gst_tracers script:

I've zipped up the files with the 1280 HEIGHT setting and attached them anyway, as requested. 

ov5647_imaging.zip

A final question so I can continue with the ISP Tuning Guide, Section "6.4.1 Launch the Tuning Tool and Create a Project" says to "Locate and run the installed tuning tool executable in Windows, \bin\DCC.exe" but where can I get the DCC.exe file from (i.e. what package download or program install provides this)? 

Thank you,

Steve

Hello Steve,

Glad that you found the problem. I missed that as well:).

The ISP tuning tool is not publicly available. You'll need to contact TI's local field applications team in order to get it. 

Regards,

Jianzhong