This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

DRA744: No surface creation during gstreamer playback

Part Number: DRA744

Hi, 

SOC : DRA746

SDK : Vision SDK 3.04

Display resolution : 800*480

Board type : Custom board

For our custom board, for carplay playback, we have below gstreamer pipeline to play the h264 stream from Iphone on the display

appsrc | ducatidecoder | waylandsink

Sometimes, we see the surface creation is not happening in the gstreamer within gstwaylandsink.

Attached are complete working and non-working case gstreamer debug logs.

Please check and help on this.

/cfs-file/__key/communityserver-discussions-components-files/791/non_2D00_working.txt

/cfs-file/__key/communityserver-discussions-components-files/791/working.txt

For non-working case : No further gstreamer logs are seen after below lines

0:00:11.242008833 2287 0x238320 DEBUG waylandsink gstwaylandsink.c:478:gst_wayland_sink_get_caps:<videoSink> display caps: video/x-raw, format=(string){ BGRA, BGRx, RGB16, YUV9, Y41B, I420, Y42B, v308, NV12, NV16, YUY2, UYVY }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:11.242129986 2287 0x238320 DEBUG waylandsink gstwaylandsink.c:523:gst_wayland_sink_set_caps:<videoSink> set caps video/x-raw, format=(string)NV12, width=(int)896, height=(int)576, framerate=(fraction)30/1, drm_mem=(boolean)true
0:00:11.242191155 2287 0x238320 DEBUG waylandsink gstwaylandsink.c:498:wait_authentication:<videoSink> Before wait aunthenticated value is 1 :
0:00:11.242224028 2287 0x238320 DEBUG waylandsink gstwaylandsink.c:503:wait_authentication:<videoSink> After wait aunthenticated value is 1 :

Best Regards,

Pradeep G

  • Hi Pradeep,

    TI' SDK did not support ivi-shell with waylandsink but I am not sure if ivi-shell here has issues.

    Can you please check if you see any error when this issue happens? I don't see any error in non-working log.

    Thanks

    Ram

  • Hi Ram, 

    yes, no errors are seen, the gst logs just stops at that point when issue is seen.

    Hence not sure if issue with ivi-shell, wayland or gstreamer here.

    Currently we are trying to capture more verbose logs for gstreamer and wayland.

    Let know if you have any inputs to check.

    Best Regards,

    Pradeep G

  • Hi ram, 

    We could reproduce the issue with GST_DEBUG=5, and Wayland debug logs.

    Attached are the complete logs : /cfs-file/__key/communityserver-discussions-components-files/791/journal2.log

    Pipeline: appsrc ! ducatih264dec ! waylandsink 

    Can you please check logs and let know on

    1. What is the error seen in ducati decoder 

    2. Surface creation is seen in wayland logs, but later no gstwaylandsink logs are present.

    3. Does gstreamer switch to appsink if waylandsink creation fails ?

    // gstreamer initialization
    Line 20740: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.549150838 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m waylandsink gstwaylandsink.c:498:wait_authentication:<videoSink>[00m Before wait aunthenticated value is 1 :
    Line 20741: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.549184219 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m waylandsink gstwaylandsink.c:503:wait_authentication:<videoSink>[00m After wait aunthenticated value is 1 :
    Line 20865: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.588257595 [336m 2237[00m 0xb36f1320 [33;01mWARN [00m [00m waylandsink wlvideoformat.c:102:gst_wl_shm_format_to_video_format:[00m gst video format not found
    Line 20867: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.588387220 [336m 2237[00m 0xb36f1320 [33;01mWARN [00m [00m waylandsink wlvideoformat.c:102:gst_wl_shm_format_to_video_format:[00m gst video format not found
    Line 20872: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.588808797 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m waylandsink gstwaylandsink.c:478:gst_wayland_sink_get_caps:<videoSink>[00m display caps: video/x-raw, format=(string){ BGRA, BGRx, RGB16, YUV9, Y41B, I420, Y42B, v308, NV12, NV16, YUY2, UYVY }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
    Line 20881: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.589428964 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m waylandsink gstwaylandsink.c:523:gst_wayland_sink_set_caps:<videoSink>[00m set caps video/x-raw, format=(string)NV12, width=(int)896, height=(int)576, framerate=(fraction)30/1, drm_mem=(boolean)true, max-ref-frames=(int)17
    Line 20882: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.589489795 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m waylandsink gstwaylandsink.c:498:wait_authentication:<videoSink>[00m Before wait aunthenticated value is 1 :
    Line 20883: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.589519617 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m waylandsink gstwaylandsink.c:503:wait_authentication:<videoSink>[00m After wait aunthenticated value is 1 :


    // below error seen in ducati decoder, what does this mean ?
    Line 20910: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.594935908 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m ducati gstducatividdec.c:567:codec_process:<vpudecoder>[00m VIDDEC3_process took 2302411ns (2 ms)
    Line 20911: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.594997247 [336m 2237[00m 0xb36f1320 [33;01mWARN [00m [00m ducati gstducatividdec.c:571:codec_process:<vpudecoder>[00m err=-1, extendedError=00040000
    Line 20912: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.595030288 [336m 2237[00m 0xb36f1320 [31;01mERROR [00m [00m ducati gstducati.c:61:gst_ducati_log_extended_error_info:[00m Bit 18 (00040000): stream end

    Line 20908: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.591708839 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m ducati gstducatividdec.c:792:gst_ducati_viddec_codec_flush:<vpudecoder>[00m flush: eos=0
    Line 20909: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.591740186 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m ducati gstducatividdec.c:822:gst_ducati_viddec_codec_flush:[00m Calling VIDDEC3_control XDM_FLUSH
    Line 20910: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.594935908 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m ducati gstducatividdec.c:567:codec_process:<vpudecoder>[00m VIDDEC3_process took 2302411ns (2 ms)
    Line 20911: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.594997247 [336m 2237[00m 0xb36f1320 [33;01mWARN [00m [00m ducati gstducatividdec.c:571:codec_process:<vpudecoder>[00m err=-1, extendedError=00040000
    Line 20912: May 12 00:10:30 mmt2020-a880 cppres[2237]: 0:10:28.595030288 [336m 2237[00m 0xb36f1320 [31;01mERROR [00m [00m ducati gstducati.c:61:gst_ducati_log_extended_error_info:[00m Bit 18 (00040000): stream end

    // preroll
    Line 21016: May 12 00:10:31 mmt2020-a880 cppres[2237]: 0:10:28.613648672 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m waylandsink gstwaylandsink.c:635:gst_wayland_sink_preroll:<videoSink>[00m preroll buffer 0x1b5f38


    // surface 90 creation
    Line 21047: May 12 00:10:31 mmt2020-a880 cppres[2237]: [1251085.891] -> ivi_application@9.surface_create(90, wl_surface@3, new id ivi_surface@17)

    // after this no waylandsink log, but appsink logs are seen, as per our media team, waylandsink creation might have failed and switched to appsink, is this right.
    Line 21052: May 12 00:10:31 mmt2020-a880 cppres[2237]: 0:10:28.617853436 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m waylandsink wldrm.c:92:gst_wl_drm_memory_construct_wl_buffer:[00m create planar buffer: 0x19b130 (name=12)
    Line 21114: May 12 00:10:31 mmt2020-a880 cppres[2237]: 0:10:28.621244188 [336m 2237[00m 0x1aa000 [37mDEBUG [00m [00m ducati gstducatividdec.c:1296:gst_ducati_viddec_src_query:<vpudecoder>[00m query: latency query: 0x1aa290, GstQueryLatency, live=(boolean)false, min-latency=(guint64)0, max-latency=(guint64)18446744073709551615;
    Line 21115: May 12 00:10:31 mmt2020-a880 cppres[2237]: 0:10:28.621319422 [336m 2237[00m 0x1aa000 [36mINFO [00m [00m ducati gstducatividdec.c:1277:gst_ducati_viddec_query:<vpudecoder>[00m latency 0:00:00.033333333 ours 0:00:00.033333333
    Line 22248: May 12 00:10:43 mmt2020-a880 cppres[2237]: 0:10:42.357466495 [336m 2237[00m 0x1aa000 [37mDEBUG [00m [00m appsink gstappsink.c:734:gst_app_sink_render:<aacsink>[00m pushing render buffer 0x1d31e8 on queue (0)
    Line 22249: May 12 00:10:43 mmt2020-a880 cppres[2237]: 0:10:42.357517328 [336m 2237[00m 0x1aa000 [37mDEBUG [00m [00m appsink gstappsink.c:1207:gst_app_sink_pull_sample:<aacsink>[00m trying to grab a buffer
    Line 22250: May 12 00:10:43 mmt2020-a880 cppres[2237]: 0:10:42.357546303 [336m 2237[00m 0x1aa000 [37mDEBUG [00m [00m appsink gstappsink.c:680:dequeue_buffer:<aacsink>[00m dequeued buffer 0x1d31e8

    // pipeline is set to play, no redraw callback seen in gstreamer logs
    Line 21068: May 12 00:10:31 mmt2020-a880 cppres[2237]: 0:10:28.618783855 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m basesink gstbasesink.c:1503:gst_base_sink_commit_state:<videoSink>[00m commiting state to PAUSED
    Line 21087: May 12 00:10:31 mmt2020-a880 cppres[2237]: 0:10:28.619796625 [336m 2237[00m 0xb36f1320 [36mINFO [00m [00;01;31m GST_STATES gstbin.c:3190:bin_handle_async_done:<dec-pipeline>[00m setting state from PAUSED to PAUSED, pending PLAYING
    Line 21099: May 12 00:10:31 mmt2020-a880 cppres[2237]: 0:10:28.620478300 [336m 2237[00m 0xb36f1320 [37mDEBUG [00m [00m basesink gstbasesink.c:2202:gst_base_sink_wait_preroll:<videoSink>[00m waiting in preroll for flush or PLAYING
    Line 21102: May 12 00:10:31 mmt2020-a880 cppres[2237]: 0:10:28.620600130 [336m 2237[00m 0x1aa000 [37mDEBUG [00m [00;01;31m GST_STATES gstbin.c:2591:gst_bin_change_state_func:<dec-pipeline>[00m changing state of children from PAUSED to PLAYING

  • Hi Pradeep,

    I analyzed the log. As soon as the decoder creation happens first frame fed doesn't have valid data. Decoder returns 0x401 error.

    ducati gstducatividdec.c:571:codec_process:<vpudecoder>[00m err=-1, extendedError=00000401
    ducati gstducati.c:61:gst_ducati_log_extended_error_info:[00m Bit 0 (00000001): no error-free slice
    ducati gstducati.c:61:gst_ducati_log_extended_error_info:[00m Bit 10 (00000400): insufficient data

    I am seeing only 32 bytes are fed to decoder as first frame and decoder doesn't find the required complete data, hence it sets no error-free slice and insufficient data errors.

    Is this happening for a specific playback of stream?

    Since this is first frame, and is not decoded, it didn't send any data to next element.

    I am not sure how appSrc works. Does this feed h264 frames to decoder?

    Thanks
    Ram

  • Hi Ram, 

    We again checked the logs, we found that, there are 2 lifecycle logs in journal logs given.

    In first lifecycle, carplay was working fine, issue was seen in 2nd lifecycle, but unfortunately initial logs of carplay connection are missing in 2nd lifecycle logs.

    So logs highlighted above are of success case scenario.

    For your questions

    > Issue is with Iphone carplay usecase, 

    > Yes, Iphone provides h264 feed which is fed to decoder by appsrc.

    We have asked again for reproduction, hopefully we get all the logs this time.

    Will update you once we have the new logs for failure case.

  • Hi Ram, 

    We could issue reproduced with GST debug level 6.

    Full logs attached here : /cfs-file/__key/communityserver-discussions-components-files/791/18may2020_2D00_journal_2D00_log_2D00_of_2D00_issue_2D00_cycle.7z

    From these logs, we still see that surface creation is not happening and no gstwaylandsink logs seen after wait_authentication, similar to what was post in the initial post.

    Can you please check logs and let know what could be reason for no surface creation.

    Log snippets

     

    // Only below gstwaylandsink logs are present. As seen earlier, after wait_authentication, gstwaylandsink logs stopped.

    Line 14277: May 12 00:12:37 mmt2020-a880 cppres[2251]: 0:12:35.078292925 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:225:shm_format:[00m shm got format: ....

    Line 14279: May 12 00:12:37 mmt2020-a880 cppres[2251]: 0:12:35.078377309 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:225:shm_format:[00m shm got format: ....

    Line 14281: May 12 00:12:37 mmt2020-a880 cppres[2251]: 0:12:35.078449492 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:225:shm_format:[00m shm got format: RG16

    Line 14285: May 12 00:12:37 mmt2020-a880 cppres[2251]: 0:12:35.078647234 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: AR24

    Line 14287: May 12 00:12:37 mmt2020-a880 cppres[2251]: 0:12:35.078717892 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: XR24

    Line 14289: May 12 00:12:37 mmt2020-a880 cppres[2251]: 0:12:35.078783806 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: YUV9

    Line 14291: May 12 00:12:37 mmt2020-a880 cppres[2251]: 0:12:35.078849889 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: YU11

    Line 14303: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.078915295 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: YU12

    Line 14305: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.078981548 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: YU16

    Line 14307: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.079050681 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: YU24

    Line 14309: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.079136250 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: NV12

    Line 14311: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.079202503 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: NV16

    Line 14313: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.079268586 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: YUYV

    Line 14315: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.079334161 [332m 2251[00m   0x240720 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:253:drm_handle_format:[00m drm got format: UYVY

    Line 14351: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.083998628 [332m 2251[00m   0x26c4f0 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:260:drm_handle_authenticated:[00m authenticated

    Line 14352: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.084055730 [332m 2251[00m   0x26c4f0 [37mDEBUG  [00m [00m         waylandsink wldisplay.c:264:drm_handle_authenticated:[00m drm_handle_authenticated: dev: 0xb3600ab0, d->authenticated: 1

    Line 15103: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.121870812 [332m 2251[00m   0x26c520 [33;01mWARN   [00m [00m         waylandsink wlvideoformat.c:102:gst_wl_shm_format_to_video_format:[00m gst video format not found

    Line 15104: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.121905210 [332m 2251[00m   0x26c520 [33;01mWARN   [00m [00m         waylandsink wlvideoformat.c:102:gst_wl_shm_format_to_video_format:[00m gst video format not found

    Line 15109: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.122264432 [332m 2251[00m   0x26c520 [37mDEBUG  [00m [00m         waylandsink gstwaylandsink.c:478:gst_wayland_sink_get_caps:<videoSink>[00m display caps: video/x-raw, format=(string){ BGRA, BGRx, RGB16, YUV9, Y41B,  420, Y42B, v308, NV12, NV16, YUY2, UYVY }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]

    Line 15128: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.123637440 [332m 2251[00m   0x26c520 [33;01mWARN   [00m [00m         waylandsink wlvideoformat.c:102:gst_wl_shm_format_to_video_format:[00m gst video format not found

    Line 15129: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.123664720 [332m 2251[00m   0x26c520 [33;01mWARN   [00m [00m         waylandsink wlvideoformat.c:102:gst_wl_shm_format_to_video_format:[00m gst video format not found

    Line 15134: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.123959723 [332m 2251[00m   0x26c520 [37mDEBUG  [00m [00m         waylandsink gstwaylandsink.c:478:gst_wayland_sink_get_caps:<videoSink>[00m display caps: video/x-raw, format=(string){ BGRA, BGRx, RGB16, YUV9, Y41B, I420, Y42B, v308, NV12, NV16, YUY2, UYVY }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]

    Line 15143: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.124595309 [332m 2251[00m   0x26c520 [37mDEBUG  [00m [00m         waylandsink gstwaylandsink.c:523:gst_wayland_sink_set_caps:<videoSink>[00m set caps video/x-raw, format=(string)NV12, width=(int)896, height=(int)576, framerate=(fraction)30/1, drm_mem=(boolean)true

    Line 15144: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.124650887 [332m 2251[00m   0x26c520 [37mDEBUG  [00m [00m         waylandsink gstwaylandsink.c:498:wait_authentication:<videoSink>[00m Before wait aunthenticated value is 1 :

    Line 15145: May 12 00:12:38 mmt2020-a880 cppres[2251]: 0:12:35.124687656 [332m 2251[00m   0x26c520 [37mDEBUG  [00m [00m         waylandsink gstwaylandsink.c:503:wait_authentication:<videoSink>[00m After wait aunthenticated value is 1 :

     

    Best Regards,

    Pradeep G

  • Hi Pradeep,

    I did some source code analysis on when preroll API is called for sink elements.

    This is triggered from basesink element when State change is happening from READY to PAUSE by sending dummy buffers for syncronization .

    This is not triggered in waylandsink case and state is not changed to PAUSE.

    This look to me it is stuck somewhere and not sending preroll buffers to waylandsink.

    Thanks

    RamPrasad