This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM67: Simple video playback not working

Part Number: AM67


Tool/software:

Hi,

We would like to play a simple video stream on v3link board without any AI components.

Do the c7x cores -need- to be running/loaded with firmware to get the cameras to work? And if the AM67 (without the A) is supposed to have the ai cores disabled, how can we use the csi camera inputs??

Here is the log we are getting:

root@j722s-evm:~# /opt/edgeai-gst-apps/scripts/setup_cameras_v3link.sh
IMX219 Camera 0 detected
device = /dev/video-imx219-cam0
name = imx219
format = [fmt:SRGGB8_1X8/1920x1080 field: none]
subdev_id = /dev/v4l-imx219-subdev0
isp_required = yes
ldc_required = yes
root@j722s-evm:~# gst-launch-1.0 v4l2src device=/dev/video-imx219-cam0 io-mode=5 ! \
> video/x-bayer,width=1920,height=1080,framerate=30/1,format=rggb ! \
> queue leaky=2 ! tiovxisp sink_0::device=/dev/v4l-imx219-subdev0 \
> sensor-name=SENSOR_SONY_IMX219_RPI dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_1920x1080.bin \
> sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_1920x1080.bin format-msb=7 \
> sink_0::pool-size=8 src::pool-size=8 ! video/x-raw,format=NV12, width=1920,height=1080 ! \
> queue ! waylandsink
APP: Init ... !!!
1275.416506 s: MEM: Init ... !!!
1275.416629 s: MEM: Initialized DMA HEAP (fd=8) !!!
1275.416790 s: MEM: Init ... Done !!!
1275.416808 s: IPC: Init ... !!!
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio2.rpmsg_chrdev.-1.13
file_deref_link: readlink failed for /sys/bus/platform/devices/7e000000.dsp
_rpmsg_char_find_rproc: 7e000000.dsp device is mostly yet to be created!
file_deref_link: readlink failed for /sys/bus/platform/devices/7e200000.dsp
_rpmsg_char_find_rproc: 7e200000.dsp device is mostly yet to be created!
1275.477460 s: IPC: ERROR: Unable to create TX channels for CPU [mcu2_0] !!!
1275.477478 s: IPC: ERROR: Unable to create TX channels for CPU [c7x_1] !!!
1275.477491 s: IPC: ERROR: Unable to create TX channels for CPU [c7x_2] !!!
1275.477504 s: IPC: Init ... Done !!!
APP: ERROR: IPC init failed !!!
REMOTE_SERVICE: Init ... !!!
_rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio2.rpmsg_chrdev.-1.21
file_deref_link: readlink failed for /sys/bus/platform/devices/7e000000.dsp
_rpmsg_char_find_rproc: 7e000000.dsp device is mostly yet to be created!
file_deref_link: readlink failed for /sys/bus/platform/devices/7e200000.dsp
_rpmsg_char_find_rproc: 7e200000.dsp device is mostly yet to be created!
REMOTE_SERVICE: Init ... Done !!!
1275.479514 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
1275.479647 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
1275.479663 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
1275.479677 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
1275.480434 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-0
1275.480632 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-1
1275.480740 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-2
1275.480836 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-3
1275.480856 s: VX_ZONE_INFO: [tivxInitLocal:126] Initialization Done !!!
1275.480872 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
Setting pipeline to PAUSED ...
1275.491616 s: VX_ZONE_ERROR: [ownContextCreateCmdObj:161] context object descriptor [0] allocation failed
1275.491675 s: VX_ZONE_ERROR: [ownContextCreateCmdObj:164] context object descriptor [0] allocation failed
1275.491692 s: VX_ZONE_ERROR: [ownContextCreateCmdObj:165] Exceeded max object descriptors available. Increase TIVX_PLATFORM_MAX_OBJ_DESC_SHM_INST value
1275.491710 s: VX_ZONE_ERROR: [ownContextCreateCmdObj:166] Increase TIVX_PLATFORM_MAX_OBJ_DESC_SHM_INST value in source/platform/psdk_j7/common/soc/tivx_platform_psdk_<soc>.h
1275.491731 s: VX_ZONE_ERROR: [vxCreateContext:1098] context objection creation failed
1275.491759 s: VX_ZONE_ERROR: [ownIsKernelInContext:677] Failed to lock context
1275.491776 s: VX_ZONE_ERROR: [ownGetErrorObject:55] Failed to lock context
1275.491792 s: VX_ZONE_ERROR: [vxGetStatus:1178] Reference is NULL
1275.491807 s: VX_ZONE_ERROR: [vxLoadKernels:209] Publish function for module openvx-core failed
1275.491840 s: VX_ZONE_ERROR: [vxAllocateUserKernelId:1787] Failed to lock context
1275.491856 s: VX_ZONE_ERROR: [tivxAddKernelDmpacSde:394] Unable to allocate user kernel ID
1275.491871 s: VX_ZONE_ERROR: [vxLoadKernels:209] Publish function for module hwa failed
1275.491892 s: VX_ZONE_ERROR: [vxAllocateUserKernelId:1787] Failed to lock context
1275.491908 s: VX_ZONE_ERROR: [tivxAddKernelDLPreProcArmv8:185] Unable to allocate user kernel ID
1275.491923 s: VX_ZONE_ERROR: [vxLoadKernels:209] Publish function for module edgeai_img_proc failed
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
1275.494716 s: VX_ZONE_ERROR: [ownIncrementReference:668] Failed to lock reference
New clock: GstSystemClock
1275.494788 s: VX_ZONE_ERROR: [ownAddReferenceToContext:385] Failed to lock context
1275.494834 s: VX_ZONE_ERROR: [ownCreateReference:770] Add reference to context failed
1275.494856 s: VX_ZONE_ERROR: [ownCreateReference:779] Failed to add to resources table
1275.494872 s: VX_ZONE_ERROR: [ownGetErrorObject:55] Failed to lock context
1275.494888 s: VX_ZONE_ERROR: [vxGetStatus:1178] Reference is NULL
1275.494905 s: VX_ZONE_ERROR: [vxGetStatus:1178] Reference is NULL

** (gst-launch-1.0:1642): CRITICAL **: 00:40:39.661: gst_tiovx_get_size_from_exemplar: assertion 'VX_SUCCESS == vxGetStatus (exemplar)' failed
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: No downstream pool to import from.
Additional debug info:
/usr/src/debug/gstreamer1.0-plugins-good/1.22.12/sys/v4l2/gstv4l2object.c(5471): gst_v4l2_object_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
When importing DMABUF or USERPTR, we need a pool to import from
Execution ended after 0:00:00.334945295
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
/usr/src/debug/gstreamer1.0/1.22.12/libs/gst/base/gstbasesrc.c(3134): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)

(gst-launch-1.0:1642): GStreamer-Video-CRITICAL **: 00:40:39.996: gst_video_info_from_caps: assertion 'caps != NULL' failed
ERROR: from element /GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0: Unable to init TIOVX module
Additional debug info:
/usr/src/debug/edgeai-gst-plugins/1.0.0/gst-libs/gst/tiovx/gsttiovxmiso.c(1512): gst_tiovx_miso_negotiated_src_caps (): /GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0
Freeing pipeline ...
1275.832584 s: VX_ZONE_ERROR: [vxUnloadKernels:258] Unable to unload kernels for module hwa
1275.832635 s: VX_ZONE_ERROR: [vxUnloadKernels:258] Unable to unload kernels for module edgeai_img_proc
1275.832682 s: VX_ZONE_ERROR: [ownDecrementReference:625] Failed to lock reference
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
1275.837401 s: IPC: ERROR: Failed to close created endpoint !!!
1275.837425 s: IPC: ERROR: Failed to close created endpoint !!!
1275.837438 s: IPC: ERROR: Failed to close created endpoint !!!
REMOTE_SERVICE: Deinit ... Done !!!
1275.837462 s: IPC: Deinit ... !!!
Caught SIGSEGV
#0 0x0000ffffa2bf0d30 in wait4 () from /usr/lib/libc.so.6
#1 0x0000ffffa2d85e6c in g_on_error_stack_trace () from /usr/lib/libglib-2.0.so.0
#2 0x0000000000405220 in ?? ()
#3 <signal handler called>
#4 0x0000ffffa2ba2814 in ?? () from /usr/lib/libc.so.6
#5 0x0000ffff9fe4d058 in appIpcDeleteRpmsgRxTask () from /usr/lib/libtivision_apps.so.10.1.0
#6 0x0000ffff9fe4c41c in appIpcDeInit () from /usr/lib/libtivision_apps.so.10.1.0
#7 0x0000ffff9fe56700 in appCommonDeInit () from /usr/lib/libtivision_apps.so.10.1.0
#8 0x0000ffffa23a9004 in ?? () from /usr/lib/libgsttiovx-1.0.so.1
#9 0x0000ffffa2ced020 in g_object_unref () from /usr/lib/libgobject-2.0.so.0
#10 0x0000ffffa23adc84 in ?? () from /usr/lib/libgsttiovx-1.0.so.1
#11 0x0000ffffa2ced020 in g_object_unref () from /usr/lib/libgobject-2.0.so.0
#12 0x0000ffffa2f18658 in ?? () from /usr/lib/libgstreamer-1.0.so.0
#13 0x0000ffffa2f17828 in gst_bin_remove () from /usr/lib/libgstreamer-1.0.so.0
#14 0x0000ffffa2f17b9c in ?? () from /usr/lib/libgstreamer-1.0.so.0
#15 0x0000ffffa2cecf1c in g_object_unref () from /usr/lib/libgobject-2.0.so.0
#16 0x0000000000404dfc in ?? ()
#17 0x0000ffffa2b484f4 in ?? () from /usr/lib/libc.so.6
#18 0x0000ffffa2b485cc in __libc_start_main () from /usr/lib/libc.so.6
#19 0x0000000000403c30 in ?? ()
Spinning. Please run 'gdb gst-launch-1.0 1642' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

  • Hi,

    Please reference this FAQ to stream from a sensor to a display: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1427869/faq-sk-am69-how-to-stream-from-csi-sensor-and-usb-camera-to-display-using-gstreamer

    Let me know if you have any further questions.

    Thank you,

    Fabiana

  • Fabiana,

    These examples all require gstreamer tiovxisp, which cannot be run on the AM67 (?)

    Are there examples accessing "device=/dev/video-imx219-cam0" without using tiovxisp?
    Could I use these devices?
    /dev/v4l/by-path/platform-30122000.ticsi2rx-video-index1
    /dev/v4l/by-path/platform-30122000.ticsi2rx-video-index2
    /dev/v4l/by-path/platform-30122000.ticsi2rx-video-index0
    /dev/v4l/by-path/platform-30122000.ticsi2rx-video-index3

  • Hi Evan,

    These examples have been validated to work on AM67. Are you using the AM67A Edge AI SDK?

    Thank you,

    Fabiana

  • Hi Fabiana,

    We are purchasing the AM67 not AM67A, so I'm intentionally leaving out the c7x memory carveouts and firmware in the device tree.
    Could you please inform us if we must buy the AM67A to support FPD Link cameras?

    Please see the image from the spec below.
    The c7x cores will not exist on the hardware we intend to purchase.



    I will confirm this setup is working after starting the c7x cores etc, however we need to understand if this entire stack is required. We do not prefer to run the entire edgeai stack if it's not needed. We would like some designs to only have 1GB of memory (The carveouts are 2GB today) and we only need two FPD Link cameras for some applications, connected only on the a53 cores, so the additional overhead and complexity in those applications is not preferred if possible.

  • Hi Evan,

    Were you unable to successfully run the pipeline below? C7x is not required to run it. Please ensure that you have a display connected beforehand.

    gst-launch-1.0 \
    v4l2src device=/dev/video-imx219-cam0 io-mode=5 ! queue leaky=2 ! \
    video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb ! \
    tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_SONY_IMX219_RPI" \
    dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_1920x1080.bin \
    sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_1920x1080.bin format-msb=7 ! \
    video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
    kmssink driver-name=tidss sync=false

    More on edgeai-gst-plugins: https://github.com/TexasInstruments/edgeai-gst-plugins/wiki

    Thank you,

    Fabiana

  • Hi Fabiana,

    Thank you for the support, but the command results in the same failure above.

    We have an LVDS screen attached to card1.
    If I specify the wrong connector-id in the kmssink gst module, then we get a different error complaining about the screen not being attached.

    I'm about to set up my hardware to enable the C7x cores.

  • Hi Evan,

    What SDK version are you using? The tiovxisp element is needed in the imx219 pipeline, because this sensor requires ISP tuning binaries.

    root@j722s-evm:~# /opt/edgeai-gst-apps/scripts/setup_cameras_v3link.sh
    IMX219 Camera 0 detected
    device = /dev/video-imx219-cam0
    name = imx219
    format = [fmt:SRGGB8_1X8/1920x1080 field: none]
    subdev_id = /dev/v4l-imx219-subdev0
    isp_required = yes
    ldc_required = yes

    Thank you,

    Fabiana

  • Hi Fabiana,

    I was missing one of the r5f memory carveouts in the device tree, causing the error message above.
    So now the stream is no longer crashing, but will hang waiting on data.

    This is a j722s yocto build 10.01.08.01. (processor-sdk-linux-10_01_08_01.txt)
    I added packages from meta-edgeai layer from processor-sdk-analytics-10.01.00-config.txt to attempt to test FPD link cameras with this later build.

    After re-enabling the c7x cores, I see the firmware loading, but it seems like perhaps my mailboxes are configured incorrectly.
    Our current overlay is not configuring the c7x mailboxes.

    APP: Init ... !!!
    20.901586 s: MEM: Init ... !!!
    20.901663 s: MEM: Initialized DMA HEAP (fd=8) !!!
    20.901851 s: MEM: Init ... Done !!!
    20.901869 s: IPC: Init ... !!!
    _rpmsg_char_find_rproc: 7e000000.dsp is either not probed or not a remoteproc!
    _rpmsg_char_find_rproc: 7e200000.dsp is either not probed or not a remoteproc!
    20.962322 s: IPC: ERROR: Unable to create TX channels for CPU [c7x_1] !!!
    20.962406 s: IPC: ERROR: Unable to create TX channels for CPU [c7x_2] !!!
    20.962422 s: IPC: Init ... Done !!!
    APP: ERROR: IPC init failed !!!
    REMOTE_SERVICE: Init ... !!!
    _rpmsg_char_find_ctrldev: could not find the matching rpmsg_ctrl device for virtio2.rpmsg_chrdev.-1.21
    _rpmsg_char_find_rproc: 7e000000.dsp is either not probed or not a remoteproc!
    _rpmsg_char_find_rproc: 7e200000.dsp is either not probed or not a remoteproc!
    REMOTE_SERVICE: Init ... Done !!!
    20.967095 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
    20.969471 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
    20.969527 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
    20.969542 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
    20.971293 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-0
    20.971686 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-1
    20.971896 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-2
    20.972067 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-3
    20.972094 s: VX_ZONE_INFO: [tivxInitLocal:126] Initialization Done !!!
    20.972110 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
    Setting pipeline to PAUSED ...
    Pipeline is live and does not need PREROLL ...
    Pipeline is PREROLLED ...
    Setting pipeline to PLAYING ...
    New clock: GstSystemClock

    Feb 27 17:59:22 j722s-evm kernel: k3-dsp-rproc 7e000000.dsp: error -19: mbox_request_channel failed
    Feb 27 17:59:22 j722s-evm kernel: remoteproc remoteproc0: releasing 7e000000.dsp
    Feb 27 17:59:22 j722s-evm kernel: k3-dsp-rproc 7e200000.dsp: error -19: mbox_request_channel failed
    Feb 27 17:59:22 j722s-evm kernel: remoteproc remoteproc0: releasing 7e200000.dsp
    Feb 27 17:59:03 j722s-evm kernel[554]: k3-dsp-rproc 7e000000.dsp: error -19: mbox_request_channel failed
    Feb 27 17:59:03 j722s-evm kernel[554]: remoteproc remoteproc0: releasing 7e000000.dsp
    Feb 27 17:59:03 j722s-evm kernel[554]: k3-dsp-rproc 7e200000.dsp: error -19: mbox_request_channel failed
    Feb 27 17:59:03 j722s-evm kernel[554]: remoteproc remoteproc0: releasing 7e200000.dsp


    I've checked the CSI pins, and I see data getting clocked into the processor.



    Update: I added the c7x mailboxes back and now the video feed is normal on our build.

    I compared our build's device tree against TI's AM67A wic image, and see that it's enabling the c7x cores with firmware and proper mailboxes.
    I pointed the c7x firmware to /dev/null and see the stream hangs like above on the evb using the regular wic image.

    So as best as I can tell, with TI's native FPD link implementation, we require the AM67A, and cannot use the AM67 with FPD link cameras?

  • Hi Fabiana,

    Would it be possible to use media-ctl to redirect the uncorrected FPD link stream to a pad which gstreamer could access without needing the tiovx module?

    Actually, I see exactly this in the document you linked earlier! So thanks again for the help.

  • Hi Evan,

    Due to a US holiday, please expect a delay in my response.

    Thank you,

    Fabiana

  • Hi Evan,

    The J722S EVM + Arudcam V3Link + IMX219/IMX390 setup was enabled and validated using AM67A Edge AI SDK. I believe your initial question has been answered, so I will be closing this thread. Please see the resources I have linked below to help you with any further questions you may have regarding this topic and let me know if there is anything else I can help you with.

    AM67A Edge AI data flows: https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-am67a/10_01_00/exports/edgeai-docs/common/edgeai_dataflows.html

    AM67 Academy: Use Camera

    [FAQ] Most common problems encountered when developing camera applications on the AM6x - Processors forum - Processors - TI E2E support forums

    Thank you,

    Fabiana

  • Hi Fabiana,

    Without the C7x cores, and using gstreamer bayer2rgb, we can only reliably get about 5 frames a second at 640x480. Uses about 50% CPU.
    Looks like we will need to map the bayer stream to some kind of glshader to decode it efficiently. However it does work.
    It's also very dark, and very blue, like the document shows.

  • Hi Evan,

    It's also very dark, and very blue, like the document shows.

    Which document are you referring to? How are you capturing the frames?

    Thank you,

    Fabiana

  • HI Fabiana,

    Earlier you linked to: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1427869/faq-sk-am69-how-to-stream-from-csi-sensor-and-usb-camera-to-display-using-gstreamer

    Which links to the "ISP Tuning guide": https://www.ti.com/lit/an/sprad86a/sprad86a.pdf

    This document describes how to do a basic check to see if the camera is working prior to using the tiovx gstreamer modules and shows them very blue.

    Here is the pipeline I used: gst-launch-1.0 -v v4l2src device=/dev/video-imx219-cam0 ! video/x-bayer, width=640, height=480, framerate=5/1, format=rggb ! bayer2rgb ! autovideosink

    Actual output, this should be mostly white:

    Supposedly we may be able to use a glshader fragment to decode and color correct the stream on the GPU without needing the C7x cores.

  • Hi Evan,

    For the best image quality, ISP tuning is always recommended. However, if you're looking to color correct using GStreamer, I can recommend using videobalance to adjust the hue in this case. The example below uses the dcc binaries, but they are not required to use the videobalance GStreamer element.

    gst-launch-1.0 \
    v4l2src device=/dev/video-imx219-cam0 io-mode=5 ! queue leaky=2 ! \
    video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb ! \
    tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_SONY_IMX219_RPI" \
    dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_1920x1080.bin \
    sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_1920x1080.bin format-msb=7 ! \
    videobalance hue="-0.5" ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
    kmssink driver-name=tidss sync=false

    Setting videobalance hue to -1.0 will "flip" the colors so that blue appears orange and green appears red. The example above is set to -0.5 so that blue appears pink and green appears purple which should help with the color imbalance you see.

    Thank you,

    Fabiana

  • Hi Fabiana,

    I confirmed we can color correct with videobalance. However, it's still CPU limited.
    We'll be looking to use gl based gstreamer modules to help with this.

    Thanks,

    Evan

  • Hi Evan,

    Ah I understand. I do not have experience using glshader, but it seems like a good choice here.

    Thanks,

    Fabiana