This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

SK-TDA4VM: Edgai-tiovx-app (Main) is not working with a specific resolution

Part Number: SK-TDA4VM

Tool/software:

Hello there TI Experts,

I am trying to use the the edgeai_tiovx_apps with my own imx568 Linux camera driver. When I launch the application I get an image like this:


As you can see it shows me something from the reality but its quite distorted. It looks like the byte order or something like that is messed up in the buffers.

In my case if I use v4l2-ctrl the image is perfectly captured. So it seems like the tiovx pipeline messes up something:

$ v4l2-ctl -d /dev/video-imx568-cam0 --set-fmt-video=width=1236,height=1032,pixelformat=RG12 --stream-mmap --stream-skip=10 --stream-count=1 --stream-to=testStream1236.raw


My format in this case is:
IMX568_CAM_FMT="${IMX568_CAM_FMT:-[fmt:SRGGB12_1X12/1236x1032]}"

To make things more interesting: My camera can also operate on 2472x2064 resolution:
IMX568_CAM_FMT="${IMX568_CAM_FMT:-[fmt:SRGGB12_1X12/2472x2064]}"

In this resolution the TIOVX application is working perfectly, just like the v4l2-ctrl. So this problem only occurs with 1236x1032, and only with edgai-apps-stack. The stream is continuous and AEWB seemingly working (stream adapts to the brightness of the room), so I think the problem is not with the DCC files.

Thank you for your help!

Zsombor Szalay

  • Hi Zsombor,

    Are you re-configuring the sensor to the desired format between capture tests? Does this sensor require ISP tuning? Each sensor configuration requires its own set of DCC binaries.

    FAQ: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1403223/faq-most-common-problems-encountered-when-developing-camera-applications-on-the-am6x

    ISP Tuning Guide: https://www.ti.com/lit/an/sprad86a/sprad86a.pdf

    Thank you,

    Fabiana

  • Hi Fabiana!

    I have generated DCCs with default_DCC_profile_gen. I haven't done ISP tuning since the default DCC-s worked pretty well with the 2472x2064 resolution (of course I generated different DCC-s for the different resolution - 1236x1032). Can this cropped, fragmented, distorted, mixed up and messed up output be the result of the lack of ISP tuning? The diagonal lines in the middle of the image which separates the mixed up segments made me think that the error has nothing to do with auto white balance and stuff like that (error is not corresponded to DCCs).

  • Hi Zsombor,

    Do you see the same distortion when running edgeai-gst-apps? I recommend running some basic GStreamer test pipelines. See this FAQ to help you build a pipeline for IMX568: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1427869/faq-sk-am69-how-to-stream-from-csi-sensor-to-display-using-gstreamer

    Thank you,

    Fabiana

  • Unfortunately  edgeai-gst-apps doesn't even starts. Here is the error what i get when I try use optiflow:

    root@tda4vm-sk:/opt/edgeai-gst-apps/optiflow# ./optiflow.py ../configs/imx568_gst.yaml 
    APP: Init ... !!!
    MEM: Init ... !!!
    MEM: Initialized DMA HEAP (fd=6) !!!
    MEM: Init ... Done !!!
    IPC: Init ... !!!
    IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
       372.665560 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
       372.665643 s:  VX_ZONE_INIT:Enabled
       372.665654 s:  VX_ZONE_ERROR:Enabled
       372.665663 s:  VX_ZONE_WARNING:Enabled
       372.666343 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0 
       372.666476 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1 
       372.666567 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2 
       372.666655 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3 
       372.666667 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
       372.667204 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
    
    (python3:1627): GStreamer-CRITICAL **: 13:05:12.388: gst_caps_get_structure: assertion 'index < GST_CAPS_LEN (caps)' failed
    
    (python3:1627): GStreamer-CRITICAL **: 13:05:12.388: gst_structure_get_int: assertion 'structure != NULL' failed
       372.689919 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
       372.690380 s:  VX_ZONE_INIT:[tivxDeInitLocal:204] De-Initialization Done !!!
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
    IPC: Deinit ... !!!
    IPC: DeInit ... Done !!!
    MEM: Deinit ... !!!
    DDR_SHARED_MEM: Alloc's: 0 alloc's of 0 bytes 
    DDR_SHARED_MEM: Free's : 0 free's  of 0 bytes 
    DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
    MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    APP: Init ... !!!
    MEM: Init ... !!!
    MEM: Initialized DMA HEAP (fd=6) !!!
    MEM: Init ... Done !!!
    IPC: Init ... !!!
    IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
       372.808615 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
       372.808760 s:  VX_ZONE_INIT:Enabled
       372.808893 s:  VX_ZONE_ERROR:Enabled
       372.808935 s:  VX_ZONE_WARNING:Enabled
       372.809367 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0 
       372.809576 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1 
       372.809765 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2 
       372.810033 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3 
       372.811296 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
       372.811401 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
    
    v4l2src device=/dev/video-imx568-cam0 io-mode=5 ! queue leaky=2 ! video/x-bayer, width=1236, height=1032, format=rggb12 ! tiovxisp sensor-name=SENSOR_SONY_IMX568 dcc-isp-file=/opt/imaging/imx568/linear/dcc_viss.bin format-msb=11 sink_0::dcc-2a-file=/opt/imaging/imx568/linear/dcc_2a.bin sink_0::device=/dev/v4l-imx568-subdev0 ! video/x-raw, format=NV12 ! \
    tiovxmultiscaler name=split_01 target=0 \
    \
    \
    split_01. ! queue ! video/x-raw, width=960, height=540 ! queue ! mosaic_0. \
    \
    \
    \
    tiovxmosaic name=mosaic_0 target=1 src::pool-size=4 \
    sink_0::startx="<320>" sink_0::starty="<150>" sink_0::widths="<960>" sink_0::heights="<540>" \
    ! video/x-raw,format=NV12, width=1920, height=1080 ! queue ! tiperfoverlay title="IMX568 Camera" overlay-type=graph ! kmssink driver-name=tidss sync=false
    
    
    (python3:1627): GStreamer-CRITICAL **: 13:05:12.608: gst_caps_get_structure: assertion 'index < GST_CAPS_LEN (caps)' failed
    
    (python3:1627): GStreamer-CRITICAL **: 13:05:12.608: gst_structure_get_int: assertion 'structure != NULL' failed
       373.493280 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
       373.497696 s:  VX_ZONE_INIT:[tivxDeInitLocal:204] De-Initialization Done !!!
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
    IPC: Deinit ... !!!
    IPC: DeInit ... Done !!!
    MEM: Deinit ... !!!
    DDR_SHARED_MEM: Alloc's: 0 alloc's of 0 bytes 
    DDR_SHARED_MEM: Free's : 0 free's  of 0 bytes 
    DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
    MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    


    Optiflow is also working fine with the same camera on 2472x2064 resolution (just like tiovx-apps). Here is the generated gst pipeline on that case:

    v4l2src device=/dev/video-imx568-cam0 io-mode=5 ! queue leaky=2 ! video/x-bayer, width=2472, height=2064, format=rggb12 ! tiovxisp sensor-name=SENSOR_SONY_IMX568 dcc-isp-file=/opt/imaging/imx568/linear/dcc_viss.bin format-msb=11 sink_0::dcc-2a-file=/opt/imaging/imx568/linear/dcc_2a.bin sink_0::device=/dev/v4l-imx568-subdev0 ! video/x-raw, format=NV12 ! \
    tiovxmultiscaler name=split_01 target=0 \
    \
    \
    split_01. ! queue ! video/x-raw, width=1280, height=720 ! queue ! mosaic_0. \
    \
    \
    \
    tiovxmosaic name=mosaic_0 target=1 src::pool-size=4 \
    sink_0::startx="<320>" sink_0::starty="<150>" sink_0::widths="<1280>" sink_0::heights="<720>" \
    ! video/x-raw,format=NV12, width=1920, height=1080 ! queue ! tiperfoverlay title="IMX568 Camera" overlay-type=graph ! kmssink driver-name=tidss sync=false

  • Hi Zsombor,

    Could you try running two test pipelines with the respective dcc binaries? Be sure to change the sensor configuration to match the test pipeline before running the command.

    IMX219 Example from the FAQ:

    gst-launch-1.0 \
    v4l2src device=/dev/video-imx219-cam0 io-mode=5 ! queue leaky=2 ! \
    video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb ! \
    tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_SONY_IMX219_RPI" \
    dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_1920x1080.bin \
    sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_1920x1080.bin format-msb=7 ! \
    video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
    kmssink driver-name=tidss sync=false

    Thank you,

    Fabiana

  • Of course imx219 is working with the gst-apps and also with the tiovx-apps (8 bit and 10 bit as well). I have used the imx219 driver as template to create my own driver. As I mentioned before imx568 is also working well with gst-app and tiovx-apps on 2472x2064 resolution. But in 1236x1032 something goes wrong. tiovx-apps gives me that messed up image and gst-apps (like optiflow) doesn't even start.

  • Hi Zsombor,

    Yes, imx219 should work as expected. I was just using the imx219 pipeline as an example, but I can create the imx568 pipelines for you. Follow step 3 of this FAQ to configure the format of the sensor to match the target resolution.

    imx568 (2472 x 2064)

    gst-launch-1.0 \
    v4l2src device=/dev/video-imx568-cam0 io-mode=5 ! queue leaky=2 ! \
    video/x-bayer, width=2472, height=2064, framerate=30/1, format=rggb12 ! \
    tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_SONY_IMX568" \
    dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_2472x2064.bin \
    sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_2472x2064.bin format-msb=7 ! \
    video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
    kmssink driver-name=tidss sync=false

    imx568 (1236 x 1032)

    gst-launch-1.0 \
    v4l2src device=/dev/video-imx568-cam0 io-mode=5 ! queue leaky=2 ! \
    video/x-bayer, width=1236, height=1032, framerate=30/1, format=rggb12 ! \
    tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_SONY_IMX568" \
    dcc-isp-file=/opt/imaging/imx568/linear/dcc_viss_1236x1032.bin \
    sink_0::dcc-2a-file=/opt/imaging/imx568/linear/dcc_2a_1236x1032.bin format-msb=7 ! \
    video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
    kmssink driver-name=tidss sync=false

    Try these and let me know if the 1236x1032 resolution is still giving you trouble.

    Thank you,

    Fabiana

  • The result in the case of imx568 (1236 x 1032) - It doesn't start:

    root@tda4vm-sk:/opt/edgeai-tiovx-apps# v4l2-ctl -d /dev/v4l-imx568-subdev0 --set-ctrl=imx568_frame_rate_control=30
    root@tda4vm-sk:/opt/edgeai-gst-apps/scripts# gst-launch-1.0 v4l2src device=/dev/video-imx568-cam0 io-mode=5 ! queue leaky=2 ! video/x-bayer, width=1236, height=1032, framerate=30/1, format=rggb12 ! tiovxisp sink_0::device=/dev/v4l-subdev0 sensor-name="SENSOR_SONY_IMX568" dcc-isp-file=/opt/imaging/imx568/linear/dcc_viss_1236x1032.bin sink_0::dcc-2a-file=/opt/imaging/imx568/linear/dcc_2a_1236x1032.bin format-msb=11 ! video/x-raw, format=NV12, width=1236, height=1032, framerate=30/1 ! kmssink driver-name=tidss sync=false
    APP: Init ... !!!
    MEM: Init ... !!!
    MEM: Initialized DMA HEAP (fd=6) !!!
    MEM: Init ... Done !!!
    IPC: Init ... !!!
    IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
      1087.809387 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
      1087.809569 s:  VX_ZONE_INIT:Enabled
      1087.809585 s:  VX_ZONE_ERROR:Enabled
      1087.809680 s:  VX_ZONE_WARNING:Enabled
      1087.810437 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0 
      1087.810708 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1 
      1087.810945 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2 
      1087.811170 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3 
      1087.811188 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
      1087.812168 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
    Setting pipeline to PAUSED ...
    Pipeline is live and does not need PREROLL ...
    Pipeline is PREROLLED ...
    Setting pipeline to PLAYING ...
    ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
    Additional debug info:
    ../gstreamer-1.20.7/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
    streaming stopped, reason not-negotiated (-4)
    ERROR: pipeline doesn't want to preroll.
    Execution ended after 0:00:00.000321939
    Setting pipeline to NULL ...
    
    (gst-launch-1.0:1804): GStreamer-CRITICAL **: 10:49:19.453: gst_caps_get_structure: assertion 'index < GST_CAPS_LEN (caps)' failed
    
    (gst-launch-1.0:1804): GStreamer-Video-CRITICAL **: 10:49:19.454: gst_video_info_from_caps: assertion 'gst_caps_is_fixed (caps)' failed
    ERROR: from element /GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0: Unable to init TIOVX module
    Additional debug info:
    ../gst-libs/gst/tiovx/gsttiovxmiso.c(1512): gst_tiovx_miso_negotiated_src_caps (): /GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0
    Freeing pipeline ...
      1087.911086 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
      1087.915581 s:  VX_ZONE_INIT:[tivxDeInitLocal:204] De-Initialization Done !!!
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
    IPC: Deinit ... !!!
    IPC: DeInit ... Done !!!
    MEM: Deinit ... !!!
    DDR_SHARED_MEM: Alloc's: 0 alloc's of 0 bytes 
    DDR_SHARED_MEM: Free's : 0 free's  of 0 bytes 
    DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
    MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    



    The result in the case of imx568 (2472 x 2064) - Works perfectly:


    root@tda4vm-sk:/opt/edgeai-gst-apps# gst-launch-1.0 v4l2src device=/dev/video-imx568-cam0 io-mode=5 ! queue leaky=2 ! video/x-bayer, width=2472, height=2064, framerate=30/1, format=rggb12 ! tiovxisp sink_0::device=/dev/v4l-subdev0 sensor-name="SENSOR_SONY_IMX568" dcc-isp-file=/opt/imaging/imx568/linear/dcc_viss_2472x2064.bin sink_0::dcc-2a-file=/opt/imaging/imx568/linear/dcc_2a_2472x2064.bin format-msb=11 ! video/x-raw, format=NV12, width=2472, height=2064, framerate=30/1 ! kmssink driver-name=tidss sync=false
    APP: Init ... !!!
    MEM: Init ... !!!
    MEM: Initialized DMA HEAP (fd=6) !!!
    MEM: Init ... Done !!!
    IPC: Init ... !!!
    IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
       214.762049 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
       214.762212 s:  VX_ZONE_INIT:Enabled
       214.762242 s:  VX_ZONE_ERROR:Enabled
       214.762257 s:  VX_ZONE_WARNING:Enabled
       214.763433 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0 
       214.763609 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1 
       214.763713 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2 
       214.763813 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3 
       214.763838 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
       214.764291 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
    Setting pipeline to PAUSED ...
    Pipeline is live and does not need PREROLL ...
    Pipeline is PREROLLED ...
    Setting pipeline to PLAYING ...
    New clock: GstSystemClock
    Redistribute latency...
    ^Chandling interrupt.
    Interrupt: Stopping pipeline ...
    Execution ended after 0:00:21.028692141
    Setting pipeline to NULL ...
    Freeing pipeline ...
       235.986485 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
       235.990928 s:  VX_ZONE_INIT:[tivxDeInitLocal:204] De-Initialization Done !!!
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
    IPC: Deinit ... !!!
    IPC: DeInit ... Done !!!
    MEM: Deinit ... !!!
    DDR_SHARED_MEM: Alloc's: 25 alloc's of 94867946 bytes 
    DDR_SHARED_MEM: Free's : 25 free's  of 94867946 bytes 
    DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
    MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    


  • Hi Zsombor,

    Could you ensure that the 1236x1032 binaries you have generated match the rggb12 format? I also recommend adding imx568 to /opt/edgeai-gst-apps/scripts/setup_cameras.sh with the SRGGB12_1X12/1236x1032 format. Once you have made the addition rerun the script and try the pipeline again.

    Thank you,

    Fabiana

  • Hi Fabiana!

    I generated the binaries with the "default_DCC_profile_gen" from the RTOS-PSDK. Here is my property file:

    SENSOR_ID 568
    PRJ_DIR ../imx568_output
    SENSOR_NAME imx568
    SENSOR_DCC_NAME IMX568
    
    SENSOR_WIDTH 1236
    SENSOR_HEIGHT 1032
    
    # 0=RGGB; 1=GRBG; 2=GBRG; 3=BGGR, 4=MONO
    COLOR_PATTERN 0
    
    # sensor mode: 0 for linear (no decompanding), 1 for WDR (decompanding)
    WDR_MODE 0
    
    # raw sensor image BIT_DEPTH: it may be 8, 10, or 12 for linear sensors; typically 12 for WDR mode because of companding
    BIT_DEPTH 12
    
    # WDR BIT_DEPTH: WDR raw sensor image bitdepth after decompanding, typically 20 or 24
    WDR_BIT_DEPTH 20
    
    # WDR decompanding knee points (comma separated without spaces in between)
    WDR_KNEE_X 0,512,1408,2176,4095,65535
    WDR_KNEE_Y 0,2048,16384,65536,1048063,1048063
    
    # Sensor black level to subtract before decompanding (for linear sensors only and some Sony WDR sensors)
    BLACK_PRE  0
    
    # Sensor black level to subtract after decompanding (for most WDR sensors and all linear sensors)
    BLACK_POST  0
    
    # GAMMA value for compressing 20/24-bit WDR raw to 16-bit ISP internal
    # typically around 50 (0.5) for 24-bit WDR sensors and 70 (0.7) for 20-bit sensors
    GAMMA_PRE 70
    
    # LSB location for H3A input bit range (from bit-H3A_INPUT_LSB to bit-H3A_INPUT_LSB+9) 
    H3A_INPUT_LSB 2
    


    Yes, I always modify the setup_cameras.sh when I change the resolution, before I even inserting the SD card to the board.

    IMX568_CAM_FMT="${IMX568_CAM_FMT:-[fmt:SRGGB12_1X12/1236x1032]}"

    I have also added my own "setup_imx568()" function to this file.
    The problem is not here I think, since capturing with v4l2-ctl is working fine, as I mentioned in the beginning.

    Best regards,

    Zsombor Szalay

  • Hi Zsombor,

    Which SDK version are you using? The reason why the 1236x1032 GStreamer pipeline won't work is because GStreamer expects width (in bytes) to be divisible by 16. This is a limitation with GStreamer and cannot be changed on our end. If you are using Linux kernel version 6.1 and beyond, you can expect to see a similar issue when using yavta capture due to a constraint being enforced by the j721e-csi2rx (SHIM) driver for width to be a multiple PSI-L word size (16 bytes).

    Thank you,

    Fabiana

  • Hi Fabiana!

    I am using Edgeai SDK 9.2.0.5. I want to use the TIOVX Apps as in my first statement, not the G-streamer Apps. I have tried GST pipelines only because you have asked to. As I know edgeai-tiovx-apps are not related to the G-streamer issue you have mentioned.

    Best regards,

    Zsombor Szalay

  • Hi Zsombor,

    You are correct. I have reached out to the development team to get an answer about the edgeai-tiovx-apps failure. I expect a similar width limitation to be present, but I will update this thread once I receive a response.

    Thank you,

    Fabiana

  • Hi Zsombor,

    Seems like this is stride miss match issue,
    Just to clarify, when you run with 1236x1032 resolution with tiovx_apps
    1. Does the application hang?
    2. Or it runs, but output is distorted

    if its option 2, then its stride issue, since 1236 is not 16 byte aligned and 2472 is aligned
    Let me get back with a fix for unaligned resolutions

    Regards
    Rahul T R

  • Hi Zsombor,

    can you please try this patch to set the alignment properly
    v4l2_capture_module: Set stride while setting v4l2 fmt · TexasInstruments/edgeai-tiovx-apps@433da8a

    Regards
    Rahul T R

  • Hello Rahul,

    Thank you for the answer and for the patch! Unfortunately the change has still not aligned the lines properly, we get the same distorted image even if I set the value for fmt.fmt.pix.bytesperline manually to 2472.

    The demo app runs, but we got the distorted image.

    Regards,

    Andras

  • Hi Rahul and Fabiana!

    I have made the changes from the git but the issue still occures. But I think you are right the mistake cold come from j721e-csi2rx driver. Do you have any idea how can I resolve this issue?

    Best regards,

    Zsombor Szalay

  • Will this 16 byte aligned constraint be solved in a future release or probably not?

  • Hi Zsombor,

    Due to the holidays, please expect a delay in response. Thank you for understanding.

    -Fabiana

  • Hi Zsombor,

    There is currently no plan to resolve this constraint. I will let comment on whether this is something that can be fixed in the future.

    I have tested this patch with IMX219 and do not see any change as well.

    Thank you,

    Fabiana

  • Hello Fabiana,

    Thank you for the answer! Could you please ask one of your feature expert colleague to point us to the right direction? Maybe we can solve this issue in house. It is still not clear for me entirely, what is the exact reason, where in the pipeline or in deeper layers do the problem rests? I assume it can be solved with code modification. It would be enough to just point us to some .c files where should we look.

    Thank you for any more information! :)

    Best Regards,

    Andras

  • Hi Andras,

    This needs to be set to 
    fmt.fmt.pix.bytesperline manually = 2480

    Can you please try this?

    Regards
    Rahul T R

  • Hello Rahul,

    Yes I have tried to to set it like this (the image resolution is 1236*1032, this is the one which does not work)

    fmt.fmt.pix.bytesperline = 1248;
    fmt.fmt.pix.sizeimage = 1248 * 1032;

    Still does not work. I assume that after receiving the image from the driver, the v4l2 api strips the padding, so the actual image resolution will be 1236*1032, but I may be wrong here.

    Best Regards,

    Andras

  • Hi Andras,

    Can you please add below line
    printf("SIZEEEEEEEEEE %llu\n", size[i]);

    at below line
    https://github.com/TexasInstruments/edgeai-tiovx-apps/blob/53eff2309ac65c17558d62f1b210cd72690e44cf/modules/core/src/tiovx_modules.c#L199

    And share the logs

    Regards
    Rahul T R

  • Hello Rahul,

    I ran my custom graph (with only one VISS node in it). Here is the log with the 1236*1032 resolution.

    APP: Init ... !!!
    MEM: Init ... !!!
    MEM: Initialized DMA HEAP (fd=5) !!!
    MEM: Init ... Done !!!
    IPC: Init ... !!!
    IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
       102.565057 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
       102.568577 s:  VX_ZONE_INIT:Enabled
       102.568607 s:  VX_ZONE_ERROR:Enabled
       102.568630 s:  VX_ZONE_WARNING:Enabled
       102.571230 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0 
       102.571368 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1 
       102.571474 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2 
       102.571644 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3 
       102.571671 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
       102.576530 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
    tiovx of node
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 1931904
    SIZEEEEEEEEEE 1931904
       106.511345 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
       106.515875 s:  VX_ZONE_INIT:[tivxDeInitLocal:204] De-Initialization Done !!!
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
    IPC: Deinit ... !!!
    IPC: DeInit ... Done !!!
    MEM: Deinit ... !!!
    DDR_SHARED_MEM: Alloc's: 13 alloc's of 26287573 bytes 
    DDR_SHARED_MEM: Free's : 13 free's  of 26287573 bytes 
    DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
    MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    

    Thanks!

    Andras

  • Hi Andras,

    Can you please try this

    fmt.fmt.pix.bytesperline = 2480;

    You can see from logs the Size is
    2559360 and 2559360/1032 is 2480
    which is the bytes used per line

    Regards
    Rahul T R

  • Hi Rahul,

    I have tried it with 2480 and also with 2496, just in case. Still the same distorted image unfortunately.

  • Hi Andras,

    Does changing this value have any effect on the level of distortion?

    Regards
    Rahul T R 

  • Hello,

    Unfortunately, no. Here is the log and the image with bytes per line value of 5600.

    APP: Init ... !!!
    MEM: Init ... !!!
    MEM: Initialized DMA HEAP (fd=5) !!!
    MEM: Init ... Done !!!
    IPC: Init ... !!!
    IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
        49.212945 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
        49.216456 s:  VX_ZONE_INIT:Enabled
        49.216541 s:  VX_ZONE_ERROR:Enabled
        49.216582 s:  VX_ZONE_WARNING:Enabled
        49.219102 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0 
        49.219259 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1 
        49.219528 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2 
        49.219649 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3 
        49.219676 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
        49.224403 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
    tiovx of node
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 2559360
    SIZEEEEEEEEEE 1931904
    SIZEEEEEEEEEE 1931904
    TIOVX BYTESPERLINE: 5600
        53.161383 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
        53.165865 s:  VX_ZONE_INIT:[tivxDeInitLocal:204] De-Initialization Done !!!
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
    IPC: Deinit ... !!!
    IPC: DeInit ... Done !!!
    MEM: Deinit ... !!!
    DDR_SHARED_MEM: Alloc's: 13 alloc's of 26287573 bytes 
    DDR_SHARED_MEM: Free's : 13 free's  of 26287573 bytes 
    DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
    MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    

    Best Regards,

    Andras

  • Hi Andras,

    Is this still an open issue?

    Best Regards,

    Fabiana

  • Hello Fabiana,

    Yes, very much so! I would appreciate any feedback regarding the matter.

    Thanks!

    Best Regards,

    Andras

  • Hi Andras,

    Can you please apply below patch to linux kernel and try
    https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/0001_2D00_j721e_2D00_csi2rx_2D00_Take_2D00_bytesperline_2D00_from_2D00_usespace.patch

    You can use PROCESSOR-SDK-LINUX-AM67A Software development kit (SDK) | TI.com (PSDK linux installer)
    to modify linux and recompile

    Also apply this in tiovx-apps
    fmt.fmt.pix.bytessperline = 2480;

    Regards
    Rahul T R

  • Hello Rahul,

    Unfortunately your change still does not take effect, see attachments.

    Regards,

    Andras

  • Hi Andras,

    on further investigation seems like driver does not support handling stride != width
    Another option is to change the stride of the OpenVX buffers, but minimum stride alignment
    required is 8 (needed by Hardware accelerators). But 1236 is not aligned to 8

    Only option to get 1236 working is doing a memcopy from CMA to openVX buffers
    does your camera supports any other resolution, which is aligned to 8?

    Regards
    Rahul T R

  • Hello Rahul,

    Yes, we have a working solution for 2472*2064, 12 bit, however we do not want to pay the cost of applying a multi-scaler module just to downsize our image.

    Could you elaborate a little on what is the optimal way to copy CMA to openVX? What are the possible costs to this solution performance wise?

    Thank you!

    Best Regards,

    Andras

  • Hi Andras,

    Do you have access to FW_BUILDER?
    I can provide a hack in VISS kernel for this
    It needs rebuilding FW, for which you need FW builder

    Regards
    Rahul T R

  • Hello Rahul,

    No, is it an open source application or part of any SDK package? I am willing to use it, so if you could provide some hack I would really appreciate it!

    Thanks in advance!

    Regards,

    Andras

  • Hi Andras,

    How did you generate DCC files for your camera?
    It is not available out of the box

    Did you use PSDK RTOS SDK?

    Regards
    Rahul T R

  • Hello Rahul,

    Yes, we use the RTOS and Linux SDK and generate the dcc file using the python tool in the RTOS SDK.

    Regards,

    Andras

  • Hi Andras,

    Below is the HACK in imaging folder of PSDK RTOS

    diff --git a/kernels/hwa/vpac_viss/vx_vpac_viss_target.c b/kernels/hwa/vpac_viss/vx_vpac_viss_target.c
    index 01c4bd6..7fa7c19 100755
    --- a/kernels/hwa/vpac_viss/vx_vpac_viss_target.c
    +++ b/kernels/hwa/vpac_viss/vx_vpac_viss_target.c
    @@ -1613,8 +1613,7 @@ static vx_status tivxVpacVissSetOutputParams(tivxVpacVissObj *vissObj,
    
                         for (cnt = 0u; cnt < TIVX_IMAGE_MAX_PLANES; cnt ++)
                         {
    -                        outPrms->fmt.pitch[cnt] =
    -                            (uint32_t)im_desc->imagepatch_addr[cnt].stride_y;
    +                        outPrms->fmt.pitch[cnt] = im_desc->width * 2;
                         }
                     }
                     else
    


    After doing this change, please follow below steps to build and install the FW to SD card

    1. cd sdk_builder; make sdk_scrub
    2. edit scripts/board_env.sh ------ change adas to edgeai
    3. edit "tools_path.mak" ------ Change PSDK_TOOLS_PATH to /home/binh/ti
                                           ------ LINUX_SD_FS_ROOT_PATH ?= /media/$(USER)/rootfs to /media/binh/rootfs
                                                   LINUX_SD_FS_BOOT_PATH ?= /media/$(USER)/BOOT to /media/binh/BOOT
    4. make sdk -j16
    4. Connect the SD card; then run   -  make linux_fs_install_sd

    Regards
    Rahul T R

  • Hello Rahul,

    I have managed to build the PSDK RTOS, here are the solutions (first with your hack, second with fixed 2480 value:

    Regards,

    Andras

  • Hi Andras,

    Here is the correct patch

    diff --git a/kernels/hwa/vpac_viss/vx_vpac_viss_target.c b/kernels/hwa/vpac_viss/vx_vpac_viss_target.c
    index 01c4bd6..6d577e3 100755
    --- a/kernels/hwa/vpac_viss/vx_vpac_viss_target.c
    +++ b/kernels/hwa/vpac_viss/vx_vpac_viss_target.c
    @@ -1688,7 +1688,7 @@ static void tivxVpacVissSetInputParams(tivxVpacVissObj *vissObj,
         /* Set the Input Format */
         fmt->width = raw_img_desc->params.width;
         fmt->height = raw_img_desc->params.height;
    -    fmt->pitch[0] = (uint32_t)raw_img_desc->imagepatch_addr[0U].stride_y;
    +    fmt->pitch[0] = fmt->width * 2;
         fmt->dataFormat = FVID2_DF_RAW;
    
         switch (raw_img_desc->params.format[0U].pixel_container)

    We should hack the input pitch not the output pitch
    Sorry for the confusion

    Can you please try with this patch.

    Regards
    Rahul T R

  • Hello Rahul,

    It's working now, thank you very much for the constant support! ;)

    Have a very nice weekend!

    Best Regards,

    Andras

  • Hi Andras,

    Thanks for letting me know :)
    Can you please mark the solution as accepted
    so that I can close the thread

    Regards
    Rahul T R