omx_videomixer to mix png image file and video file

Hello,

Rajnikant Jachak said:

Is omx_videomixer immature element in EZSDK-5.3.1.15 ?

Hello Margarita,

I tried latest ezsdk (5_05_02_00) on dm814x EVM board. But now board is not booting up.

Following are the boot logs which I captured from my minicom terminal:

Mon Dec 31 10:37:00 UTC 2012
INIT: Entering runlevel: 5
Loading HDVICP2 Firmware
DM814X prcm_config_app version: 2.0.0.1
Doing PRCM settings...
        PRCM for IVHD0 is in Progress, Please wait.....  
                        BW Phy Addr : 0x48180600 Data : 0x00000001
                        AW Phy Addr : 0x48180600 Data : 0x00000002
                        Phy Addr : 0x48180c04 Data : 0x00000037
                Data : 0x00000007
                        AW Phy Addr : 0x48180c10 Data : 0x00000003
                        Phy Addr : 0x48180c14 Data : 0x00000004
                        BW Phy Addr : 0x58088000 Data : 0xd17dcf5c
                        AW Phy Addr : 0x58088000 Data : 0xeafffffe
                        BW Phy Addr : 0x58098000 Data : 0x8fd19924
                        AW Phy Addr : 0x58098000 Data : 0xeafffffe
                        BW Phy Addr : 0x48180c10 Data : 0x00000003
                        AW Phy Addr : 0x48180c10 Data : 0x00000000
                        Phy Addr : 0x48180c14 Data : 0x00000007
        PRCM for IVHD0 is Done Successfully  
PRCM Initialization completed
SysLink version : 2.20.02.20
SysLink module created on Date:Dec 26 2012 Time:18:39:49
FIRMWARE: I2cInit will be done by M3
FIRMWARE: Memory map bin file not passed
Usage : firmware_loader <Processor Id> <Location of Firmware> <start|stop> [-mmap <memory_map_file>] [-i2c <0|1>]
===Mandatory arguments===
<Processor Id>         0: DSP, 1: Video-M3, 2: Vpss-M3
<Location of Firmware> firmware binary file
<start|stop>           to start/stop the firmware
===Optional arguments===
-mmap                  input memory map bin file name
-i2c                   0: i2c init not done by M3, 1(default): i2c init done by M3
FIRMWARE: isI2cInitRequiredOnM3: 1
FIRMWARE: Default memory configuration is used
Firmware Loader debugging not configured
Default FL_DEBUG: warning
Allowed FL_DEBUG levels: error, warning, info, debug, log
MemCfg: DCMM (Dynamically Configurable Memory Map) Version :  2.1.2.1
vmap allocation for size 197136384 failed: use vmalloc=<size> to increase size.
Unable to handle kernel paging request at virtual address 40000010
pgd = ef1b0000
[40000010] *pgd=af09b031, *pte=00000000, *ppte=00000000
Internal error: Oops: 817 [#1]
last sysfs file: /sys/kernel/uevent_seqnum
Modules linked in: syslink ipv6
CPU: 0    Not tainted  (2.6.37 #1)
PC is at HeapMemMP_postInit+0x178/0x228 [syslink]
LR is at GateMP_getSharedAddr+0x4c/0x60 [syslink]
pc : [<bf099d84>]    lr : [<bf095f38>]    psr: 80000013
sp : ef18bd78  ip : 00000000  fp : ef18bdb4
r10: 00000001  r9 : ef18be14  r8 : ef18be00
r7 : f0807000  r6 : f080a000  r5 : 40000000  r4 : 0bbfff80
r3 : 00000580  r2 : 00000580  r1 : 00060000  r0 : 00000580
Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
Control: 10c5387d  Table: af1b0019  DAC: 00000015
Process firmware_loader (pid: 2054, stack limit = 0xef18a2e8)
Stack: (0xef18bd78 to 0xef18c000)
bd60:                                                       00000002 00000004
bd80: ef18bdb4 ef18bd90 ef18bdb4 ef18bd98 bf083264 bf074358 40000000 f080a000
bda0: f0807000 00000000 ef18bdf4 ef18bdb8 bf09b63c bf099c18 ef18be00 00000001
bdc0: ef18bdf4 00000000 bf0809ac ef18be00 ef18be44 f0b51048 00000000 00000002
bde0: 00000048 00000000 ef18be2c ef18bdf8 bf09bfa0 bf09b250 f0b51048 00000002
be00: 00000000 00000000 40000000 0bbfff80 00000000 f0807000 0bc00000 ef18be44
be20: ef18be84 ef18be30 bf085764 bf09be74 ef18beb8 ef1b1338 ef18bea4 f0b51000
be40: 00000001 00000000 00000000 40000000 0bbfff80 00000000 bf075630 c024f2a0
be60: eee19280 be80aa64 00000009 be80aa64 ef18a000 00000000 ef18bef4 ef18be88
be80: bf0c8218 bf085488 c024f2a0 be80aa64 ef18beac ef18bea0 c00bc1dc c00b3214
bea0: ef18becc ffffffff 41266000 b3d00000 b3d00000 be80aa9c 40000000 0bc00000
bec0: 00010003 00000000 00000080 00000001 00000000 00000000 00000000 eee19280
bee0: 00000009 00000009 ef18bf04 ef18bef8 c00d76ec bf0c7dcc ef18bf74 ef18bf08
bf00: c00d7dfc c00d76d0 000000fb 000000fb 41266000 ef348c80 ef123180 0bc00000
bf20: 0000bc00 000b3d00 ef18bf74 ef18bf38 c00be258 c00bdbe0 000000fb 000b3d00
bf40: ef18a000 ef348c80 00000001 00000000 be80aa64 c024f2a0 00000009 eee19280
bf60: ef18a000 00000000 ef18bfa4 ef18bf78 c00d7e94 c00d7908 00000001 00000001
bf80: 00000003 00046ee8 00000000 00008e20 00000036 c0047fa8 00000000 ef18bfa8
bfa0: c0047e00 c00d7e48 00046ee8 00000000 00000009 c024f2a0 be80aa64 00000009

eth0: no IPv6 routers present

The boot process did not proceed after 'eth0: no IPv6 routers present' message.

Could you please suggest how to resolve this?

Thanks,

Rajnikant

Hello Rajnikant,

Rajnikant Jachak said:

vmap allocation for size 197136384 failed: use vmalloc=<size> to increase size.
Unable to handle kernel paging request at virtual address 40000010
pgd = ef1b0000
[40000010] *pgd=af09b031, *pte=00000000, *ppte=00000000
Internal error: Oops: 817 [#1]

Following are bootargs

bootargs=console=ttyO0,115200n8 rootwait rw mem=364M@0x80000000 mem=320M@0x9FC00000 root=/dev/nfs nfsroot=10.1.1.243: ip=dhcp

Thanks

Hi Margarita,

I have upgraded EZSDK to 'ezsdk_5_05_02_00' version and tried using omx_videomixer for following pipeline, but it gave error.

gst-launch  -v \
    omx_videomixer name=mix \
    ! omx_ctrl \
    ! omx_videosink sync=true \
    filesrc location=M2959_1_ENG_3_5.png \
    ! pngdec ! ffmpegcolorspace  \
    ! imagefreeze \
    ! mix. \
    filesrc location=Dabang_4_1080p24_unscrambled_H264.mp4 ! qtdemux \
    ! queue ! h264parse ! omx_h264dec ! gstperf print-arm-load=1 ! omx_scaler \
    ! ffmpegcolorspace \
    ! mix.

--------------------------------- ERROR ------------------------------------

(gst-launch-0.10:1555): GLib-GObject-WARNING **: invalid cast from `GstBuffer' to `GstOmxBufferTransport'
Caught SIGSEGV accessing address (nil)
perf0: frames: 15       current: 6.06    average: 6.06  arm-load: 11
Spinning.  Please run 'gdb gst-launch 1555' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

So do I still need to patch omx? If yes where should I find patch?

Thanks,

Hello,

I will check it.

Rajnikant Jachak said:

gst-launch  -v \
    omx_videomixer name=mix \
    ! omx_ctrl \
    ! omx_videosink sync=true \
    filesrc location=M2959_1_ENG_3_5.png \
    ! pngdec ! ffmpegcolorspace  \
    ! imagefreeze \
    ! mix. \
    filesrc location=Dabang_4_1080p24_unscrambled_H264.mp4 ! qtdemux \
    ! queue ! h264parse ! omx_h264dec ! gstperf print-arm-load=1 ! omx_scaler \
    ! ffmpegcolorspace \
    ! mix.

Could you try :

gst-launch omx_videomixer name=mix  ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! gstperf ! omx_videosink  filesrc location=sample2.mp4 ! qtdemux name=demux demux.audio_00 ! queue ! faad ! alsasink demux.video_00 ! queue ! h264parse ! omx_h264dec  ! mix.           ....    

Best Regards,

Margarita

Hello,

You could check and this pipeline with videomixer:

gst-launch --gst-debug=3 --verbose omx_videomixer framerate=15 port-index=0 name=mix ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! gstperf ! omx_videosink sync=false filesrc location=/usr/share/ti/data/videos/dm816x_1080p_demo.264 ! 'video/x-h264' ! h264parse access-unit=true ! omx_h264dec ! mix. filesrc location=/usr/share/ti/data/videos/dm816x_1080p_demo.264 ! 'video/x-h264' ! h264parse access-unit=true ! omx_h264dec ! mix. filesrc location=/usr/share/ti/data/videos/dm816x_1080p_demo.264 ! 'video/x-h264' ! h264parse access-unit=true !  omx_h264dec ! gstperf ! mix.

Best Regards,

Margarita

Thanks Margarita.

The above pipeline gave me following error:

------------------------------------- ERROR ----------------------------------

/GstPipeline:pipeline0/GstOmxH264Dec:omxh264dec0.GstPad:src: caps = video/x-raw-yuv-strided, width=(int)1920, height=(int)1080, format=(fourcc)NV12, rowstride=(int)2048, interlaced=(boolean)false, framerate=(fraction)0/1
OMX_EventError: 0x80001000
unrecoverable error: There were insufficient resources to perform the requested operation (0x80001000)
0:00:01.763601601  1513 0x503009e0 WARN                     omx gstomx_base_filter.c:708:pad_chain:<omxh264dec2> error: Error from OpenMAX component
ERROR: from element /GstPipeline:pipeline0/GstOmxH264Dec:omxh264dec2: GStreamer encountered a general stream error.
Additional debug info:
gstomx_base_filter.c(708): pad_chain (): /GstPipeline:pipeline0/GstOmxH264Dec:omxh264dec2:
Error from OpenMAX component
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
0:00:01.765686351  1513 0x503009e0 WARN                 basesrc gstbasesrc.c:2574:gst_base_src_loop:<filesrc2> error: Internal data flow error.
0:00:01.765787451  1513 0x503009e0 WARN                 basesrc gstbasesrc.c:2574:gst_base_src_loop:<filesrc2> error: streaming task paused, reason error (-5)

I checked with 2 video input for mixer and it ran wel with 299M memory and avg 45% CPU.

I want to display subtitles on my video and my subtitles are image files (png format). So I am thinking of using omx_videomixer to mix these two files and get desired effect.

Can  I achieve this using omx_videomixer? we have achieved same by using following pipeline on linux desktop:

gst-launch  -v \
    videomixer name=mix sink_1::xpos=520 sink_1::ypos=960 sink_1::alpha=0.3 sink_1::zorder=3 sink_2::xpos=400 sink_2::ypos=100 sink_2::zorder=2 background=3 \
    ! xvimagesink sync=true \
    filesrc location=M2959_1_ENG_3_5.png \
    ! pngdec ! queue ! ffmpegcolorspace \
    ! imagefreeze !  mix. \
    filesrc location=Dabang_4_1080p24_unscrambled_H264.mp4 \
    ! qtdemux ! queue \
    ! legacyh264parse ! queue ! ffdec_h264 \
    ! ffmpegcolorspace \
    ! mix.

Thanks

Hello,

This pipeline works for me.

Margarita Gashova said:

gst-launch --gst-debug=3 --verbose omx_videomixer framerate=15 port-index=0 name=mix ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! gstperf ! omx_videosink sync=false filesrc location=/usr/share/ti/data/videos/dm816x_1080p_demo.264 ! 'video/x-h264' ! h264parse access-unit=true ! omx_h264dec ! mix. filesrc location=/usr/share/ti/data/videos/dm816x_1080p_demo.264 ! 'video/x-h264' ! h264parse access-unit=true ! omx_h264dec ! mix. filesrc location=/usr/share/ti/data/videos/dm816x_1080p_demo.264 ! 'video/x-h264' ! h264parse access-unit=true !  omx_h264dec ! gstperf ! mix.

perf0: frames: 84     current: 31.99     average: 27.62    arm-load: 46
perf1: frames: 101     current: 32.00     average: 25.03    arm-load: 47
perf0: frames: 117     current: 32.01     average: 28.73    arm-load: 43
perf1: frames: 133     current: 31.87     average: 26.39    arm-load: 42
perf0: frames: 149     current: 31.80     average: 29.34    arm-load: 47

Rajnikant Jachak said:

GstPipeline:pipeline0/GstOmxH264Dec:omxh264dec0.GstPad:src: caps = video/x-raw-yuv-strided, width=(int)1920, height=(int)1080, format=(fourcc)NV12, rowstride=(int)2048, interlaced=(boolean)false, framerate=(fraction)0/1
OMX_EventError: 0x80001000
unrecoverable error: There were insufficient resources to perform the requested operation (0x80001000)
0:00:01.763601601  1513 0x503009e0 WARN                     omx gstomx_base_filter.c:708:pad_chain:<omxh264dec2> error: Error from OpenMAX component
ERROR: from element /GstPipeline:pipeline0/GstOmxH264Dec:omxh264dec2: GStreamer encountered a general stream error.
Additional debug info:
gstomx_base_filter.c(708): pad_chain (): /GstPipeline:pipeline0/GstOmxH264Dec:omxh264dec2:
Error from OpenMAX component
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
0:00:01.765686351  1513 0x503009e0 WARN                 basesrc gstbasesrc.c:2574:gst_base_src_loop:<filesrc2> error: Internal data flow error.
0:00:01.765787451  1513 0x503009e0 WARN                 basesrc gstbasesrc.c:2574:gst_base_src_loop:<filesrc2> error: streaming task paused, reason error (-5)

Could you post your bootargs?

Rajnikant Jachak said:

I want to display subtitles on my video and my subtitles are image files (png format). So I am thinking of using omx_videomixer to mix these two files and get desired effect.

Can  I achieve this using omx_videomixer?

I will check it and I will let you know the result.

Best Regards,

Margarita

These are my bootargs:

console=ttyO0,115200n8 rootwait rw mem=256M earlyprintk notifyk.vpssm3_sva=0xBF900000 vram=50M ti814xfb.v
ram=0:16M,1:16M,2:6M root=/dev/nfs nfsroot=10.1.1.243:/opt/targetfs ip=dhcp

Thanks

Hello,

Rajnikant Jachak said:

console=ttyO0,115200n8 rootwait rw mem=256M earlyprintk notifyk.vpssm3_sva=0xBF900000 vram=50M ti814xfb.v
ram=0:16M,1:16M,2:6M root=/dev/nfs nfsroot=10.1.1.243:/opt/targetfs ip=dhcp

What are you using DM814 or DM816?

Rajnikant Jachak said:

gst-launch  -v \
    omx_videomixer name=mix \
    ! omx_ctrl \
    ! omx_videosink sync=true \
    filesrc location=M2959_1_ENG_3_5.png \
    ! pngdec ! ffmpegcolorspace  \
    ! imagefreeze \
    ! mix. \
    filesrc location=Dabang_4_1080p24_unscrambled_H264.mp4 ! qtdemux \
    ! queue ! h264parse ! omx_h264dec ! gstperf print-arm-load=1 ! omx_scaler \
    ! ffmpegcolorspace \
    ! mix.

--------------------------------- ERROR ------------------------------------

(gst-launch-0.10:1555): GLib-GObject-WARNING **: invalid cast from `GstBuffer' to `GstOmxBufferTransport'
Caught SIGSEGV accessing address (nil)
perf0: frames: 15       current: 6.06    average: 6.06  arm-load: 11
Spinning.  Please run 'gdb gst-launch 1555' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

In this case, the using of ffmpegcolorspace component leads to this error in yours pipeline.

I will continue to investigate.

Best Regards,

Margarita

Thanks Margarita for continued attention.

I'm using DM1848 EVM and EZSDK version 5_05_02_00.

Thanks

Hello Rajnikant Jachak,

You could use videomixer element:

gst-launch  --gst-debug=3  videomixer name=mix ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60  ! omx_videosink sync=true     filesrc location=12664924405.png   ! pngdec ! queue !  ffmpegcolorspace   ! imagefreeze !  mix.

Is it working for you?

Let me know the result.

Best Regards,

Margarita

Hello Margarita,

I had tried videomixer element with following pipeline:

gst-launch  -v \
    videomixer name=mix sink_1::xpos=520 sink_1::ypos=960 sink_1::alpha=0.3 sink_1::zorder=3 sink_2::xpos=400 sink_2::ypos=100 sink_2::zorder=2 \
    ! omx_ctrl display-device=HDMI \
    ! omx_videosink display-device=HDMI sync=true \
    filesrc location=M2959_1_ENG_3_5.png \
    ! pngdec ! queue ! ffmpegcolorspace \
    ! imagefreeze \
    ! mix. \
    filesrc location=Dabang_4_1080p24_unscrambled_H264.mp4 ! qtdemux \
    ! h264parse ! omx_h264dec ! queue ! gstperf print-arm-load=1 ! omx_scaler \
    ! ffmpegcolorspace \
    ! mix.

----------------- gstperf stats ---------------------------

perf0: frames: 16       current: 15.99   average: 15.99 arm-load: 20
perf0: frames: 20       current: 3.73    average: 9.65  arm-load: 100
perf0: frames: 24       current: 3.89    average: 7.74  arm-load: 99
perf0: frames: 28       current: 3.85    average: 6.76  arm-load: 100

This pipeline achieves image overlay on video. But as you can see in gstperf statistics, the framerate drops drastically for videomixer on EVM (Original is 24 FPS).

I moved to omx_videomixer as I think omx uses hardware acceleration to speed up the processing.

Hello Rajnikant Jachak,

The frame rate dropped not because of using on omx_videomixer or videomixer element, it is dropped because of ffmpegcolorspace elements in the pipeline.

ffmpegcolorspace — converts video from one colorspace to another.

I will continue to investigate.

Best Regards,

Margarita