SK-TDA4VM: How can I enable and debug IMX390, DS90UB953 and DS90UB960 with Edge AI SDK?

Hi,

Which port on your fusion board is the IMX390 sensor connected to? The IMX390 overlay used depends on its position where the following for k3-j721e-fpdlink-imx390-rcm-<x>-<y>.dtbo is true:

CSI lane the camera is connected to: <x> = {0, 1}

Position on fusion board: <y> = {0, 1, 2, 3}

Thanks,

Fabiana

The IMX390 sensor is connected to port 0 on the fusion board. Since the Sk board and fusion board have only one Samtec Interface, so connect them directly. Therefore, I can't determine if the csi lane the camera is connected to is 0 or 1. However, as shown in the image above, I have tried both csi lanes and failed to detect imx390.

Could you try using k3-j721e-fpdlink-imx390-cm-0-0.dtbo?

Thank you,

Fabiana

I tried using k3-j721e-fpdlink-imx390-cm-0-0.dtbo, and the result was similar to that when using k3-j721e-fpdlink-imx390-rcm-0-0.dtbo. The difference was that the I2C address of IMX390 was different. The following is the printed information of the output.
Also, I wonder if the current problem is caused by the fact that PROCESSOR-SDK-LINUX-SK-TDA4VM(v08.06.01.03) is not compatible with IMX390 of the built-in ISP?

Thank you very much for your reply. Through reading the linked article you provided, I have a general understanding of the current problem. I still need your answers to the following four questions, looking forward to receiving your reply. 

1. I want to know what the difference is between rcm and cm. When setting the name_overlays variable in uEnv.txt file, should xxx in k3-j721-fpdlink-imx390-xxx-0-0.dtbo be rcm or cm? In the linked article you provided above, I found that the customer set rcm, but the sensor driver patch that was finally provided appears to be cm. 

2. Regarding the patch provided in the above link, does it work with my current processor, SDK, fusion board and Camera(details are explained at the beginning of the question)? If not, can you provide the corresponding patch? 

3. Currently, I also use SDK: PROCESSOR-SDK-LINUX-SK-TDA4VM(v09.02.00.05), and the problems I encountered are shown in the following figure. Besides not supporting IMX390 with built-in ISP, are there any other possible errors? ds90ub960 cannot be detected because it has an error different from v08.06.01.03. I would appreciate it if you could give me some answers. 

4. The details of the last question are shown in the link. e2echina.ti.com/.../sk-tda4vm-sdk-processor-sdk-linux-sk-tda4vm-v09x-09 - 02-05-08-APr-2024?tisearch=e2e-sitesearch&keymatch=SK-TDA4VM#, this question was raised a month ago but has not been answered. I hope to get your help. Thank you.

Hi,

I believe CM stands for camera module and RCM stands for rugged camera module. Most commonly, customers use the D3RCM-IMX390-953 sensor which is rcm.

The patch should be applicable to this newer SDK as there haven't been any changes to the IMX390 driver between these SDK releases.

Based on the pictures you have shared, I do not believe the issue is with the ds90ub960 driver. I took a look at the link you shared and it appears to be an issue with GStreamer pipelines. Are you experiencing a similar issue?

Thank you,

Fabiana

First of all, thank you for your reply, which has brought great help to my work. As for the third question, I think my previous expression is wrong, and I don't think this is the problem of ds90ub960 driver. After doing the same thing with the hardware connection and adding.dtbo to the uEnv.txt file, the information printed looks like this, and there seems to be one more ds90ub960 related issue in it compared to using version 8.6 of the SDK. I don't know what caused this, but I didn't do any pipeline building.
Also, regarding the fourth question, it has nothing to do with the third question because I am using videotestsrc and not v4l2src as the data source, looking forward to your reply.

Hello,

I'm glad to help! The driver for ds90ub960 was updated after v8.6 to add support for imx219 formats. Besides this, the driver functions the same and these messages you see are not indicative of any issues.

Using videotestsrc as your data source is supported. Recently, we had a customer report an issue with using snow pattern but this issue is not observed in our latest SDK and a patch is available in this thread if you do not wish to use a newer SDK: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1318105/tda4vh-q1-v4l2h265enc-cannot-work-with-videotestsrc-when-pattern-is-snow/5055510

If you run into any issues with your GStreamer pipelines, feel free to ask a new question about this!

Thank you,

Fabiana

Hello, sorry to reply you after so long.
Based on the PROCESSOR-SDK-LINUX-SK-TDA4VM Linux SDK(v09.02.00.05), we tried using the patch provided in the following thread, e2e.ti.com/.../tda4vm-some-errors-when-using-imx390-from-otobrite-on-linux.
And made the following modifications to the dts file:

As shown below, the imx390 sensor was not successfully detected.

After checking, I found that the sensor used by the camera module in the above thread is different from the sensor used by our camera module, so the result is that the patch cannot be effectively used.

Can you provide us with an effective patch that will enable us to successfully detect the imx390 sensor?

In addition, please recommend a camera module compatible with SK-TDA4VM+TIDA-01413, thank you, looking forward to your reply.

Hello,

Please allow me some time to look into this for you.

Thanks,

Fabiana

Thank you for your support, and as time is tight, I would appreciate your reply as soon as possible.

Hi,

Due to bandwidth and this specific sensor being outside our support, this is not something we can help you with. I suggest you use the previous patch and port it over to the kernel version you are using yourself.

As far as other supported sensors on SK-TDA4VM go, IMX390 without built-in ISP use DS90UB960 and DS90UB953 deserializer and serializer. We validate this sensor using the Fusion1 Rev. C daughter board, but any board that uses compatible deser/der should work fine as well.

Thank you,

Fabiana

Hi, I used the following command to build a gstreamer pipeline and successfully captured the image. However, except that the color of the first frame is correct, the color of each subsequent frame is skewed to a red tone. I don't know what the problem is, but I would appreciate it if you could give me some useful advice.

gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=dmabuf-import ! video/x-bayer, width=1936, height=1100, framerate=30/1, format=rggb12 ! tiovxisp sink_0::device=/dev/v4l-subdev4 sensor-name="SENSOR_SONY_IMX390_UB953_D3" dcc-isp-file=/opt/imaging/imx390/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/imx390/dcc_2a.bin format-msb=9 ! video/x-raw, format=NV12, width=1936, height=1100, framerate=30/1 ! tiovxmultiscaler ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! jpegenc ! multifilesink location=”/home/root/imx390/%d.jpg”

SDK: PROCESSOR-SDK-LINUX-SK-TDA4VM (v08.06.01.03)

Host board: SK-TDA4VM

Fusion board: TIDA-01413 (Deserializer: DS90UB960-Q1)

Camera: SG2-IMX390C-FPDLink-H120L (Serializer: DS90UB953)

Hi,

Because the sensor you are using has built-in ISP, the color issue may be due to the inclusion of the ISP binaries. Try excluding these from your pipeline.

Thank you,

Fabiana

Thank you so much.However, I have already replaced the sensor before and the current sensor does not have a built-in ISP module

Hi,

Can you confirm that you see a similar message to the one below upon boot? If so, could you share that output?

IMX219 Camera 1 detected
    device = /dev/video-imx219-cam0
    name = imx219
    format = [fmt:SRGGB8_1X8/1920x1080]
    subdev_id = /dev/v4l-imx219-subdev0
    isp_required = yes

If this message is missing, please share the output of media-ctl -p.

Thank you,

Fabiana

Yes, I see a message upon boot.

 IMX390 Camera 0 detected

 device=/dev/video2

 name=imx390

 format=[fmt:SRGGB12_1*12/1936*1100 field: none]

 subdev_id =/dev/v4l-subdev4

 isp_required=yes

 ldc_required=yes

Hi,

Try running the following:

gst-launch-1.0 \
v4l2src device=/dev/video2 io-mode=dmabuf-import ! \
video/x-bayer, width=1936, height=1100, format=rggb12 ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 sensor-name=SENSOR_SONY_IMX390_UB953_D3 \
dcc-isp-file=/opt/imaging/imx390/linear/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/imx390/linear/dcc_2a.bin format-msb=11 ! \
tiovxldc dcc-file=/opt/imaging/imx390/linear/dcc_ldc.bin sensor-name=SENSOR_SONY_IMX390_UB953_D3 sink_0::pool-size=8 src::pool-size=8 ! \
video/x-raw, format=NV12, width=1936, height=1100 ! kmssink driver-name=tidss sync=false

Please include any logs if you come across an issue. If the color still looks off, follow the tuning procedure outlined in this ISP tuning guide: https://www.ti.com/lit/an/sprad86a/sprad86a.pdf

Thanks,

Fabiana

The configuration file you are using cannot be found:
/ opt/imaging/imx390 / linear/dcc_viss. Bin,
/ opt/imaging/imx390 / linear/dcc_2a. Bin,
/ opt/imaging/imx390 /linear/dcc_ldc. Bin,
Please provide us with these configuration files.

We only found the following profile:
/opt/imaging/imx390/dcc_viss.bin,
/opt/imaging/imx390/dcc_2a.bin,
/opt/imaging/imx390/dcc_ldc.bin,
Using the commands we described earlier, the resulting image frame colors are as follows, only the first two frames of the image frame color is normal, and all the image frames after the color is red