Other Parts Discussed in Thread: TDA4VM
Hello,
We are using a gst pipeline to stream camera frames to test our ai models. The pipeline that we used so far was working without any problem but after appending tiovxldc & tiovxmultiscaler to our pipeline we observed some random crushes as attached debug output. Meaning that we run the pipeline several times without any problem and after some point it does not start due to this error (requires reboot to solve). It seems the problem is related to VPAC hwa itself but we couldn't be sure.
Could you please enlighten us regarding the details? What could be the reason behind that? How to solve this issue without rebooting?
The Pipeline: GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" gst-launch-1.0 v4l2src device=/dev/video2 ! tiovxdlcolorconvert ! video/x-raw, format=NV12 ! tiovxldc dcc-file=/home/root/TDA4VM_mesh_ldc_dcc.bin sensor-name=SENSOR_SONY_IMX390_UB953_D3 ! tiovxmultiscaler name=msc src_0::roi-startx=320 src_0::roi-starty=280 src_0::roi-width=1280 src_0::roi-height=480 target=1 interpolation-method=16385 msc. ! video/x-raw, format=NV12, width=1024, height=384 ! videoconvert ! videoflip method=horizontal-flip ! waylandsink
APP: Init ... !!! MEM: Init ... !!! MEM: Initialized DMA HEAP (fd=9) !!! MEM: Init ... Done !!! IPC: Init ... !!! IPC: Init ... Done !!! REMOTE_SERVICE: Init ... !!! REMOTE_SERVICE: Init ... Done !!! 2649.829616 s: GTC Frequency = 200 MHz APP: Init ... Done !!! 2649.830750 s: VX_ZONE_INIT:Enabled 2649.830778 s: VX_ZONE_ERROR:Enabled 2649.830828 s: VX_ZONE_WARNING:Enabled 2649.831431 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0 2649.831622 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1 2649.831744 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2 2649.831852 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3 2649.831864 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!! 2649.832407 s: VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!! Requested pad src_0 2649.864735 s: VX_ZONE_ERROR:[ownContextSendCmd:875] Command ack message returned failure cmd_status: -7 2649.864771 s: VX_ZONE_ERROR:[ownNodeKernelInit:590] Target kernel, TIVX_CMD_NODE_CREATE failed for node node_109 2649.864865 s: VX_ZONE_ERROR:[ownNodeKernelInit:591] Please be sure the target callbacks have been registered for this core 2649.864899 s: VX_ZONE_ERROR:[ownNodeKernelInit:592] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel 2649.864933 s: VX_ZONE_ERROR:[ownGraphNodeKernelInit:608] kernel init for node 0, kernel com.ti.hwa.vpac_ldc ... failed !!! 2649.864992 s: VX_ZONE_ERROR:[vxVerifyGraph:2159] Node kernel init failed 2649.865001 s: VX_ZONE_ERROR:[vxVerifyGraph:2213] Graph verify failed 2649.865159 s: VX_ZONE_ERROR:[ownReleaseReferenceInt:598] Invalid reference 2650.009072 s: VX_ZONE_ERROR:[vxGetStatus:1020] Reference is NULL ** (gstreamer_example:3100): CRITICAL **: 13:53:40.145: gst_tiovx_simo_process_graph: assertion 'VX_SUCCESS == vxGetStatus ((vx_reference) priv->graph)' failed 2650.022711 s: VX_ZONE_WARNING:[vxReleaseContext:1325] Found a reference 0xffff8a499388 of type 00000816 at external count 1, internal count 0, releasing it 2650.022779 s: VX_ZONE_WARNING:[vxReleaseContext:1327] Releasing reference (name=user_data_object_102) now as a part of garbage collection 2650.022970 s: VX_ZONE_WARNING:[vxReleaseContext:1325] Found a reference 0xffff8a4995b0 of type 00000816 at external count 1, internal count 0, releasing it 2650.022981 s: VX_ZONE_WARNING:[vxReleaseContext:1327] Releasing reference (name=user_data_object_103) now as a part of garbage collection 2650.023073 s: VX_ZONE_WARNING:[vxReleaseContext:1325] Found a reference 0xffff8a5b1f58 of type 00000813 at external count 1, internal count 0, releasing it 2650.023083 s: VX_ZONE_WARNING:[vxReleaseContext:1327] Releasing reference (name=object_array_105) now as a part of garbage collection 2650.023183 s: VX_ZONE_WARNING:[vxReleaseContext:1325] Found a reference 0xffff8a4de538 of type 0000080f at external count 1, internal count 0, releasing it 2650.023194 s: VX_ZONE_WARNING:[vxReleaseContext:1327] Releasing reference (name=image_106) now as a part of garbage collection 2650.023255 s: VX_ZONE_WARNING:[vxReleaseContext:1325] Found a reference 0xffff8a5b2100 of type 00000813 at external count 1, internal count 0, releasing it 2650.023264 s: VX_ZONE_WARNING:[vxReleaseContext:1327] Releasing reference (name=object_array_107) now as a part of garbage collection 2650.023321 s: VX_ZONE_WARNING:[vxReleaseContext:1325] Found a reference 0xffff8a4de800 of type 0000080f at external count 1, internal count 0, releasing it 2650.023330 s: VX_ZONE_WARNING:[vxReleaseContext:1327] Releasing reference (name=image_108) now as a part of garbage collection MEM: ERROR: Failed to free memory at virt addr = 0xffff7c7cd000 !!! 2650.023392 s: VX_ZONE_ERROR:[tivxMemBufferFree:262] Shared ptr mem free failed 2650.023434 s: VX_ZONE_ERROR:[ownDestructImage:273] Image buffer free failed! 2650.023465 s: VX_ZONE_ERROR:[ownReleaseReferenceInt:578] destructor() returned error. 2650.023496 s: VX_ZONE_ERROR:[vxReleaseContext:1360] Failed to destroy external reference objects 2650.023526 s: VX_ZONE_ERROR:[vxReleaseContext:1378] Reference 113 not removed 2650.023621 s: VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!! 2650.023673 s: VX_ZONE_ERROR:[ownObjectDeInit:231] Is image use failed, index: 6 2650.028065 s: VX_ZONE_INIT:[tivxDeInitLocal:204] De-Initialization Done !!! APP: Deinit ... !!! REMOTE_SERVICE: Deinit ... !!! REMOTE_SERVICE: Deinit ... Done !!! IPC: Deinit ... !!! IPC: DeInit ... Done !!! MEM: Deinit ... !!! DDR_SHARED_MEM: Alloc's: 17 alloc's of 38052448 bytes DDR_SHARED_MEM: Free's : 17 free's of 38052448 bytes DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes MEM: Deinit ... Done !!! APP: Deinit ... Done !!!