AM62A7: VPAC-related GStreamer plugins in SBL boot mode

Hi Jason,

Can you please explain what you did to utilize SBL boot, instead of the normal boot flow as illustrated here: software-dl.ti.com/.../UG-General-Info.html

Thanks,

Jianzhong

First, we incorporated the modifications for the ROI issue in ti-firmware-builder-am62axx-evm-10_00_00_05/imaging/kernels/hwa/vpac_msc/vx_vpac_msc_multi_scale_output_target.c within the firmware builder. Following the documentation, we ultimately compiled the following files.

DMR5 firmware -> vision_apps/out/AM62A/R5F/FREERTOS/release/vx_app_rtos_linux_mcu1_0_strip.out

c7x firmware -> vision_apps/out/AM62A/C7504/FREERTOS/release/vx_app_rtos_linux_c7x_1.out

tispl.bin -> ${LINUX_INSTALL_PATH}/board-support/ti-u-boot-*/am62a-arm64-linux/tispl.bin

Then, to optimize boot time, we trimmed down uboot and spl, replacing them with SBL boot. Following the SBL boot process manual, we compiled several appimages and flashed them using the following commands. Among them, dm_r5.appimage was compiled using mcu_plus_sdk_am62ax_10_00_00_14/examples/drivers/ipc/ipc_rpmsg_echo_linux/am62ax-sk/r5fss0-0_freertos.

echo 0 > /sys/block/mmcblk0boot0/force_ro

dd if=/mnt/sbl_emmc/sbl_emmc_linux_stage1.tiimage of=/dev/mmcblk0boot0 seek=0

dd if=/mnt/sbl_emmc/sbl_emmc_linux_stage2.appimage of=/dev/mmcblk0boot0 seek=1024

dd if=/mnt/sbl_emmc/dm_r5.appimage of=/dev/mmcblk0boot0 seek=1536

dd if=/mnt/sbl_emmc/hsm_m4.appimage of=/dev/mmcblk0boot0 seek=4608

dd if=/mnt/sbl_emmc/mcu_r5.appimage of=/dev/mmcblk0boot0 seek=8192

dd if=/mnt/sbl_emmc/linux.appimage.hs_fs of=/dev/mmcblk0boot0 seek=20480

However, we found that directly using this appimage caused GStreamer in the Linux system to fail and report errors related to mcu1_0 and c7x. To resolve these two issues, we made the following modifications.

1. Through documentation, we discovered that VPAC is loaded via dm r5f. The reason for the inability to use VPAC-related plugins was suspected to be that the ipc_rpmsg_echo_linux demo was not linked to VPAC-related content. Therefore, we modified the demo's Makefile, copied over vx_app_rtos_linux_mcu1_0.out, and linked this .out file, ultimately compiling an ipc_rpmsg_echo_linux.release.appimage. After flashing this dm_r5f.appimage, the mcu1_0 error disappeared.
2. Next was the c7x issue. We initially intended to follow the same approach used to fix mcu1_0 by modifying and linking vx_app_rtos_linux_c7x_1.out into ipc_rpmsg_echo_linux.release.appimage. However, we encountered a new problem: vx_app_rtos_linux_c7x_1.out was too large, at a full 20MB, and our Linux Image was already 20MB. Our eMMC's boot0 partition is only 32MB in size, making it impossible to fit both files simultaneously. In the end, we removed the dsp_c75 allocation from the boot0 partition and found that we could still load the c75 firmware after Linux booted.

With these changes, GStreamer could start normally without any mcu1_0 or c7x errors.

However, after this migration, a new issue emerged. When I tested the following GStreamer command, everything worked normally:

# normal
gst-launch-1.0 videotestsrc is-live=true pattern=0 ! \
video/x-raw,width=1920,height=1536,format=NV12,framerate=60/1 ! \
queue ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
kmssink driver-name=tidss sync=false skip-vsync=true

But when I added an additional output after ​msc, the entire screen turned green.

# bug
gst-launch-1.0 videotestsrc is-live=true pattern=0 ! \
video/x-raw,width=1920,height=1536,format=NV12,framerate=60/1 ! \
queue ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
multi.src_2 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! fakesink \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
kmssink driver-name=tidss sync=false skip-vsync=true

Hi Jianzhong

AM62A7: Gstreamer color format error when splitting video stream - Processors forum - Processors - TI E2E support forums

After testing, I found that the issue I mentioned earlier still persists and has been successfully reproduced on the EVM as well. Here are the test commands:

gst-launch-1.0 videotestsrc is-live=true pattern=blue ! \
video/x-raw,width=1920,height=1536,format=NV12,framerate=60/1 ! \
queue ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
kmssink driver-name=tidss sync=false skip-vsync=true

BR,

Jason

Hi Jason,

Thanks for the update. Your SBL boot procedure looks good.

I'll look into the ROI issue with our SW team.

Regards,

Jianzhong

Hi Jianzhong

Is there any update with this ROI issue?

I've noticed that the color erro issue occurring after integrating VPAC is more severe than the previous color problem caused by modifying the multiscaler  plugin to enable parallel Y&UV processing. The previous color issue only appeared when the MSC was connected to two outputs - it would disappear with three or more MSC outputs. But this time, any configuration with more than two outputs is abnormal.

gst-launch-1.0 videotestsrc is-live=true pattern=blue ! \
video/x-raw,width=1920,height=1536,format=NV12,framerate=60/1 ! \
queue ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
src_2::roi-startx=0 src_2::roi-starty=0 src_2::roi-width=1920 src_2::roi-height=1080 \
src_3::roi-startx=0 src_3::roi-starty=0 src_3::roi-width=1920 src_3::roi-height=1080 \
src_4::roi-startx=0 src_4::roi-starty=0 src_4::roi-width=1920 src_4::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
multi.src_2 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! fakesink \
multi.src_3 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! fakesink \
multi.src_4 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! fakesink \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
kmssink driver-name=tidss sync=false skip-vsync=true

BR,

Jason

Hi Jason,

This issue may be caused by tiovxmosaic. Can you try to use a single image as input and dump each ROI output to a file and check if the output files look good?

Thank you.

Jianzhong