• State TI Thinks Resolved
  • Locked Locked
  • Replies 10 replies
  • Answers 1 answer
  • Subscribers 100 subscribers
  • Views 311 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

SK-AM62A-LP: tiovxisp getting failed to output GRAY8 data

Jaimin Patel
Intellectual 650 points
Part Number: SK-AM62A-LP

Tool/software:

Hello Team,

We are using below gstreamer pipeline for GRAY8 data. camera sensor provided monochrome data and that we are trying to pass in ISP using bayer format but ISP getting failed to output GRAY8 data.

Below pipeline failed for GRAY8 output in tiovxisp:

gst-launch-1.0 -v v4l2src device=/dev/video-rpi-cam0 io-mode=dmabuf-import ! video/x-bayer, width=1920, height=1200, framerate=60/1, format=rggb ! tiovxisp sink_0::device=/dev/v4l-rpi-subdev0 sensor-name="SENSOR_SONY_IMX219_RPI" dcc-isp-file=/opt/imaging/ar0235/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/ar0235/dcc_2a.bin format-msb=9 ! video/x-raw, format=GRAY8, wid
th=1920, height=1200 ! fakesink

Working pipeline for NV12 output in tiovxisp:

gst-launch-1.0 -v v4l2src device=/dev/video-rpi-cam0 io-mode=dmabuf-import ! video/x-bayer, width=1920, height=1200, framerate=60/1, format=rggb ! tiovxisp sink_0::device=/dev/v4l-rpi-subdev0 sensor-name="SENSOR_SONY_IMX219_RPI" dcc-isp-file=/opt/imaging/ar0235/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/ar0235/dcc_2a.bin format-msb=9 ! video/x-raw, format=NV12, wid
th=1920, height=1200 ! fakesink

Error log:


root@am62axx-evm:/usr/sbin#
root@am62axx-evm:/usr/sbin# gst-launch-1.0 -v v4l2src device=/dev/video-rpi-cam0 io-mode=dmabuf-import ! video/x-bayer, width=1920, height=1200, framerate=60/1, format=rggb ! tiovxisp sink_0::device=/dev/v4l-rpi-subdev0 sensor-name="SENSOR_SONY_IMX219_RPI" dcc-isp-file=/opt/imaging/ar0235/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/ar0235/dcc_2a.bin format-msb=9 ! video/x-raw, format=GRAY8, wid
th=1920, height=1200 ! fakesink
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
8935.369247 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
8935.369359 s: VX_ZONE_INIT:Enabled
8935.369374 s: VX_ZONE_ERROR:Enabled
8935.369389 s: VX_ZONE_WARNING:Enabled
8935.370185 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
8935.370316 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1200, framerate=(fraction)60/1, format=(string)rggb, interlace-mode=(string)progressive, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1200, framerate=(fraction)60/1, format=(string)rggb, interlace-mode=(string)progressive, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0.GstTIOVXIspPad:sink_0: caps = video/x-bayer, width=(int)1920, height=(int)1200, framerate=(fraction)60/1, format=(string)rggb, interlace-mode=(string)progressive, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-bayer, width=(int)1920, height=(int)1200, framerate=(fraction)60/1, format=(string)rggb, interlace-mode=(string)progressive, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0.GstTIOVXMisoPad:src: caps = video/x-raw, format=(string)GRAY8, width=(int)1920, height=(int)1200
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, format=(string)GRAY8, width=(int)1920, height=(int)1200
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw, format=(string)GRAY8, width=(int)1920, height=(int)1200
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, format=(string)GRAY8, width=(int)1920, height=(int)1200
8935.749007 s: VX_ZONE_ERROR:[ownReleaseReferenceInt:601] Invalid reference
ERROR: from element /GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0: Unable to init TIOVX module
Additional debug info:
../git/gst-libs/gst/tiovx/gsttiovxmiso.c(1508): gst_tiovx_miso_negotiated_src_caps (): /GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0
Execution ended after 0:00:00.370444909
Setting pipeline to NULL ...
Freeing pipeline ...
8936.051869 s: VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff9ee95350 of type 00000816 at external count 1, internal count 0, releasing it
8936.051920 s: VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=user_data_object_108) now as a part of garbage collection
8936.052065 s: VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff9ee95570 of type 00000816 at external count 1, internal count 0, releasing it
8936.052092 s: VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=user_data_object_109) now as a part of garbage collection
8936.052176 s: VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff9ef352f8 of type 00000813 at external count 1, internal count 0, releasing it
8936.052202 s: VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=object_array_111) now as a part of garbage collection
8936.052249 s: VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff9ee959b0 of type 00000816 at external count 1, internal count 0, releasing it
8936.052272 s: VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=user_data_object_112) now as a part of garbage collection
8936.052307 s: VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff9ef35498 of type 00000813 at external count 1, internal count 0, releasing it
8936.052330 s: VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=object_array_113) now as a part of garbage collection
8936.052364 s: VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff9eea3700 of type 00000817 at external count 1, internal count 0, releasing it
8936.052386 s: VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=raw_image_114) now as a part of garbage collection
8936.052420 s: VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff9ef35638 of type 00000813 at external count 1, internal count 0, releasing it
8936.052443 s: VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=object_array_115) now as a part of garbage collection
8936.052478 s: VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff9ee95bd0 of type 00000816 at external count 1, internal count 0, releasing it
8936.052500 s: VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=user_data_object_116) now as a part of garbage collection
8936.052534 s: VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff9ef357d8 of type 00000813 at external count 1, internal count 0, releasing it
8936.052556 s: VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=object_array_117) now as a part of garbage collection
8936.052590 s: VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff9eec7708 of type 0000080f at external count 1, internal count 0, releasing it
8936.052613 s: VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=image_118) now as a part of garbage collection
8936.052768 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
8936.057370 s: VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
IPC: Deinit ... !!!
IPC: DeInit ... Done !!!
MEM: Deinit ... !!!
DDR_SHARED_MEM: Alloc's: 20 alloc's of 18823304 bytes
DDR_SHARED_MEM: Free's : 20 free's of 18823304 bytes
DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes
MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!
root@am62axx-evm:/usr/sbin#
root@am62axx-evm:/usr/sbin#
root@am62axx-evm:/usr/sbin#




Any idea why ISP failed to output GRAY8 ? 


Thanks,
Jaimin

  • 0
    TI__Guru 52685 points

    Hello Jaimin,

    GRAY8 output format is meant for IR processing by the ISP. Please refer to https://github.com/TexasInstruments/edgeai-gst-plugins/blob/main/ext/tiovx/gsttiovxisp.c#L1157.I believe that's why your pipeline with GRAY8 failed.

    Regards,

    Jianzhong

  • 0 in reply to Jianzhong Xu
    Intellectual 650 points

    Thanks for update Jianzhong. 

  • 0 in reply to Jaimin Patel
    Intellectual 650 points

    Hi Jianzhong,

    ISP is accepting input video format as bayer. So we are passing monochrome data in ISP through RGGB format. However we are not able to get GRAY8 data from ISP. Is there any way available to fetch GRAY8 data from ISP?

    If GRAY8 data is not possible from ISP then can we achieve below through any way?

    Here received NV12(YUV) data contains image information in all three components (Y, U and V). However monochrome image should only have Y data. Is there any why such that ISP provide only Y data and U/V marked as 0?

    Thanks,

  • 0 in reply to Jaimin Patel
    Intellectual 650 points

    In addition to above comment we tried to dump ISP processed NV12 image from ISP and we observed that U and V components of YUV contains 0x80 in all bytes. Does these U and V component required to re-create image if we convert NV12 in to GRAY8 format?

  • 0 in reply to Jaimin Patel
    TI__Guru 52685 points

    Hi Jaimin,

    Let me investigate if and how we can get GRAY8 output for monochrome input. Please ping me if you don't hear me in a couple of days.

    Thanks,

    Jianzhong

  • 0 in reply to Jianzhong Xu
    Intellectual 650 points

    Hi Jianzhong,

    Did you get a chance to check the way to get GRAY8 output from ISP?  Additionally I found in below link indicates we can get GRAY8 and GRAY16 both data from tiovxisp plugin.

    https://github.com/TexasInstruments/edgeai-gst-plugins/wiki/tiovxisp

    But not sure why we are unable to get image data in GRAY8 and GRAY16 format. Please let us know.

    Thanks 

  • 0 in reply to Jaimin Patel
    TI__Guru 52685 points

    Hi Jamin,

    We don't have the API to output Y data only without U&V. You can try to use the gst plugin "videoconvert" to convert NV12 to GRAY8.

    Jaimin Patel said:
    Additionally I found in below link indicates we can get GRAY8 and GRAY16 both data from tiovxisp plugin.

    That is meant for RGB-IR data processing.

    Regards,

    Jianzhong

  • 0 in reply to Jianzhong Xu
    Intellectual 650 points

    Hi Jianzhong,

    Jaimin Patel said:
    In addition to above comment we tried to dump ISP processed NV12 image from ISP and we observed that U and V components of YUV contains 0x80 in all bytes. Does these U and V component required to re-create image if we convert NV12 in to GRAY8 format?

    All data bytes in U and V components contains 0x80 value when we pass monochrome input to ISP. Is this image data? Or it is dummy and we can ignore  them?

    Jianzhong Xu said:
    That is meant for RGB-IR data processing.

    Does that mean we can not fetch GRAY8/GRAY16 data from ISP while providing input format in monochrome?

    Thanks

  • 0 in reply to Jaimin Patel
    TI__Guru 52685 points
    Jaimin Patel said:
    All data bytes in U and V components contains 0x80 value when we pass monochrome input to ISP. Is this image data? Or it is dummy and we can ignore  them?

    What did you configure the ISP to process for the monochrome data? 

    Jaimin Patel said:
    Does that mean we can not fetch GRAY8/GRAY16 data from ISP while providing input format in monochrome?

    Yes, GRAY8 output is only available for RGB-IR input.

    Regards,

    Jianzhong

  • 0 in reply to Jianzhong Xu
    TI__Guru 52685 points

    Hello Jaimin,

    Jaimin Patel said:
    All data bytes in U and V components contains 0x80 value when we pass monochrome input to ISP. Is this image data? Or it is dummy and we can ignore  them?

    U and V components being 0x80 means this is mono chrome data. To confirm, you can capture a raw input image and YUV output image. They should be very close to each other.

    To get GRAY8 output for mono chrome input, the SoC is capable, but the SW is not ready yet. We'll need to update the Gstreamer plugin gsttiovxisp and the underlining TIOVX. We currently don't have a plan to do that. In your application, I hope you can read Y data only from the YUV output buffer.

    Regards,

    Jianzhong