Linux/TDA2EVM5777: Display controller registration fails with object detection usecase

Ian43425

Part Number: TDA2EVM5777

Tool/software: Linux

I have added the following usecase to $VISION_SDK/apps/src/hlos/adas/src/usecases directory (named lvds_object_detection).

Capture -> Alg_ImagePreProcess (EVE1) -> Dup_capture -> Merge_alg

Dup_capture -> Alg_ImgPyramid -> Dup_imgPyramid -> Alg_FPCompute (EVE1) -> Alg_ObjectDetection (DSP1) -> Merge_objectDetect (DSP1)

Dup_imgPyramid -> Merge_objectDetect (DSP1) -> Sync_objectDetect (DSP2) -> Alg_ObjectClassification (DSP2) -> Merge_alg -> Sync_alg -> Alg_ObjectDraw -> SgxFrmcpy (A15)

// display on m4
SgxFrmcpy -> Display_alg (IPU1_0)

GrpxSrc -> Display_Grpx

This is essentially taken (and modified) from the object detection usecase that resides in the 'rtos' directory but since I am wanting to use linux, I figured I would need to create a usecase in the 'hlos' directory.

Everything compiles and the sample app is created with my usecase in the menu.

But when I run it, the Display fails to open. See output of console below.

I thought that the linkid may have been incorrect, but have confirmed that it looks correct based on the linkid of a previous usecase that I have successfully been able to implement.

I have examined the code at the lines specified (dispcore/src/vpscore_dss.c @ Line 465) and cannot see an obvious reason for my issue.

Before diving in and trying to determine where the issue is, I thought I would ask here to see if there is something obvious that I am doing wrong.

So my questions are

1. Should the usecase I have generated work as-is or have I made a mistake in the description?

2. Is there any way to enable extra output help to debug this?

3. Is there anything else obvious that you can see that is wrong here?

Thanks,

Ian

[HOST] [IPU2 ] 48697.812694 s: CAPTURE: Create in progress !!!
[HOST] [IPU2 ] 48697.813060 s: CAPTURE: VIP1 Slice0 PortA capture mode is [ 8-bit] !!!
[HOST] [EVE1 ] 48697.935094 s: IPC_IN_0 : Create in progress !!!
[HOST] [EVE1 ] 48697.936314 s: IPC_IN_0 : Create Done !!!
[HOST] [IPU2 ] 48697.933477 s: CAPTURE: Create Done !!!
[HOST] [IPU2 ] 48697.934148 s: IPC_OUT_0 : Create in progress !!!
[HOST] [IPU2 ] 48697.934484 s: IPC_OUT_0 : Create Done !!!
[HOST] [EVE1 ] 48697.937076 s: ALGORITHM: Create in progress (algId = 10) !!!
[HOST] [EVE1 ] 48697.957664 s: ALGORITHM: Create Done (algId = 10) !!!
[HOST] [EVE1 ] 48697.958640 s: IPC_OUT_0 : Create in progress !!!
[HOST] [EVE1 ] 48697.959159 s: IPC_OUT_0 : Create Done !!!
[HOST] [IPU2 ] 48697.959738 s: IPC_IN_0 : Create in progress !!!
[HOST] [IPU2 ] 48697.961019 s: IPC_IN_0 : Create Done !!!
[HOST] [IPU2 ] 48697.962483 s: ALGORITHM: Create in progress (algId = 5) !!!
[HOST] [IPU2 ] 48697.963338 s: ALG_IMAGEPYRAMID: # 0 Input 0,246 : 1280 x 230 (1280 B) (In Org 1280 x 720)
[HOST] [IPU2 ] 48697.963551 s: Output 0,244 : 1280 x 228 (1280 B) (Out Org 1280 x 720)
[HOST] [IPU2 ] 48697.963795 s: ALG_IMAGEPYRAMID: # 1 Input 0,222 : 1278 x 270 (1280 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.963948 s: Output 0,188 : 1076 x 228 (1088 B) (Out Org 1076 x 604)
[HOST] [IPU2 ] 48697.964100 s: ALG_IMAGEPYRAMID: # 2 Input 0,196 : 1278 x 322 (1280 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.964253 s: Output 0,140 : 904 x 228 ( 928 B) (Out Org 904 x 508)
[HOST] [IPU2 ] 48697.964405 s: ALG_IMAGEPYRAMID: # 3 Input 0,160 : 1278 x 382 (1280 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.964558 s: Output 0, 96 : 760 x 228 ( 768 B) (Out Org 760 x 428)
[HOST] [IPU2 ] 48697.964741 s: ALG_IMAGEPYRAMID: # 4 Input 0, 0 : 1280 x 720 (1280 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.964893 s: Output 0, 0 : 640 x 360 ( 640 B) (Out Org 640 x 360)
[HOST] [IPU2 ] 48697.965046 s: ALG_IMAGEPYRAMID: # 5 Input 0, 42 : 636 x 270 ( 640 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.965168 s: Output 0, 36 : 536 x 228 ( 544 B) (Out Org 538 x 302)
[HOST] [IPU2 ] 48697.965320 s: ALG_IMAGEPYRAMID: # 6 Input 0, 16 : 638 x 322 ( 640 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.965473 s: Output 0, 12 : 452 x 228 ( 480 B) (Out Org 452 x 254)
[HOST] [IPU2 ] 48697.965625 s: ALG_IMAGEPYRAMID: # 7 Input 0, 0 : 638 x 356 ( 640 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.965808 s: Output 0, 0 : 380 x 212 ( 384 B) (Out Org 380 x 214)
[HOST] [IPU2 ] 48697.965961 s: ALG_IMAGEPYRAMID: # 8 Input 0, 0 : 640 x 360 ( 640 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.966113 s: Output 0, 0 : 320 x 180 ( 320 B) (Out Org 320 x 180)
[HOST] [IPU2 ] 48697.966266 s: ALG_IMAGEPYRAMID: # 9 Input 0, 0 : 318 x 176 ( 320 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.966418 s: Output 0, 0 : 268 x 148 ( 288 B) (Out Org 268 x 150)
[HOST] [IPU2 ] 48697.966540 s: ALG_IMAGEPYRAMID: #10 Input 0, 0 : 316 x 174 ( 320 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.966754 s: Output 0, 0 : 224 x 124 ( 224 B) (Out Org 226 x 126)
[HOST] [IPU2 ] 48697.966906 s: ALG_IMAGEPYRAMID: #11 Input 0, 0 : 316 x 174 ( 320 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.967059 s: Output 0, 0 : 188 x 104 ( 192 B) (Out Org 190 x 106)
[HOST] [IPU2 ] 48697.967211 s: ALG_IMAGEPYRAMID: #12 Input 0, 0 : 320 x 180 ( 320 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.967364 s: Output 0, 0 : 160 x 88 ( 160 B) (Out Org 160 x 90)
[HOST] [IPU2 ] 48697.967486 s: ALG_IMAGEPYRAMID: #13 Input 0, 0 : 156 x 84 ( 160 B) (In Org 0 x 0)
[HOST] [IPU2 ] 48697.967699 s: Output 0, 0 : 132 x 72 ( 160 B) (Out Org 134 x 74)
[HOST] [IPU2 ] 48698.158390 s: ALGORITHM: Create Done (algId = 5) !!!
[HOST] [DSP1 ] 48698.160556 s: IPC_IN_1 : Create in progress !!!
[HOST] [DSP1 ] 48698.161013 s: IPC_IN_1 : Create Done !!!
[HOST] [DSP1 ] 48698.166717 s: IPC_IN_0 : Create in progress !!!
[HOST] [DSP1 ] 48698.167510 s: IPC_IN_0 : Create Done !!!
[HOST] [DSP1 ] 48698.167693 s: ALGORITHM: Create in progress (algId = 8) !!!
[HOST] [EVE1 ] 48698.162203 s: IPC_IN_1 : Create in progress !!!
[HOST] [EVE1 ] 48698.163453 s: IPC_IN_1 : Create Done !!!
[HOST] [EVE1 ] 48698.164185 s: ALGORITHM: Create in progress (algId = 11) !!!
[HOST] [EVE1 ] 48698.164765 s: ALGORITHM: Create Done (algId = 11) !!!
[HOST] [EVE1 ] 48698.165680 s: IPC_OUT_1 : Create in progress !!!
[HOST] [EVE1 ] 48698.166198 s: IPC_OUT_1 : Create Done !!!
[HOST] [IPU2 ] 48698.159915 s: IPC_OUT_2 : Create in progress !!!
[HOST] [IPU2 ] 48698.160251 s: IPC_OUT_2 : Create Done !!!
[HOST] [IPU2 ] 48698.161288 s: IPC_OUT_1 : Create in progress !!!
[HOST] [IPU2 ] 48698.161623 s: IPC_OUT_1 : Create Done !!!
[HOST] [DSP1 ] 48698.173092 s: ALGORITHM: Create Done (algId = 8) !!!
[HOST] [DSP1 ] 48698.173549 s: IPC_OUT_0 : Create in progress !!!
[HOST] [DSP1 ] 48698.173641 s: IPC_OUT_0 : Create Done !!!
[HOST] [DSP2 ] 48698.173824 s: IPC_IN_0 : Create in progress !!!
[HOST] [DSP2 ] 48698.174129 s: IPC_IN_0 : Create Done !!!
[HOST] [DSP2 ] 48698.174525 s: ALGORITHM: Create in progress (algId = 17) !!!
[HOST] [DSP2 ] 48698.177545 s: ALGORITHM: Create Done (algId = 17) !!!
[HOST] [DSP2 ] 48698.177758 s: IPC_OUT_0 : Create in progress !!!
[HOST] [DSP2 ] 48698.177850 s: IPC_OUT_0 : Create Done !!!
[HOST] [IPU2 ] 48698.178094 s: IPC_IN_1 : Create in progress !!!
[HOST] [IPU2 ] 48698.179161 s: IPC_IN_1 : Create Done !!!
[HOST] [IPU2 ] 48698.180900 s: ALGORITHM: Create in progress (algId = 1) !!!
[HOST] [IPU2 ] 48698.182089 s: UTILS: DMA: Allocated CH (TCC) = 32 (32)
[HOST] [IPU2 ] 48698.182211 s: UTILS: DMA: 0 of 1: Allocated PaRAM = 32 (0x63304800)
[HOST] [HOST ] 1453.607319 s: IPC_IN_0 : Create in progress !!!
[HOST] [HOST ] 1453.607746 s: IPC_IN_0 : Create Done !!!
[HOST] [HOST ] 1453.607807 s: SGXFRMCPY: Create in progress for resolution: 1920x1080 !!!
[HOST] [HOST ] 1453.608478 s: SYSTEM_PVRSCOPE: FIFO /opt/vision_sdk/gpufifo opened successfully
[HOST] [IPU2 ] 48698.246507 s: ALGORITHM: Create Done (algId = 1) !!!
[HOST] [IPU2 ] 48698.246995 s: IPC_OUT_3 : Create in progress !!!
[HOST] [IPU2 ] 48698.247331 s: IPC_OUT_3 : Create Done !!!
failed to load module: /usr/lib/gbm/gbm_dri.so: cannot open shared object file: No such file or directory
failed to load module: /usr/lib/gbm/gbm_gallium_drm.so: cannot open shared object file: No such file or directory
loaded module : gbm_pvr.so
found valid GBM backend : gbm_pvr.so
[HOST] [HOST ] 1453.664569 s: EGL: version 1.4
[HOST] [HOST ] 1453.702817 s: EGL: GL Version = OpenGL ES 2.0 build 1.14@3699939 (MAIN)
[HOST] [HOST ] 1453.702817 s: EGL: GL Vendor = Imagination Technologies
[HOST] [HOST ] 1453.702817 s: EGL: GL Renderer = PowerVR SGX 544MP
[HOST] [HOST ] 1453.702817 s: EGL: GL Extensions = GL_OES_rgb8_rgba8 GL_OES_depth24 GL_OES_vertex_half_float GL_OES_texture_float GL_OES_texture_half_float GL_OES_element_index_uint GL_OES_mapbuffer GL_OES_fragment_precision_high GL_OES_compressed_ETC1_RGB8_texture GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_required_internalformat GL_OES_depth_texture GL_OES_get_program_binary GL_OES_packed_depth_stencil GL_OES_standard_derivatives GL_OES_vertex_array_object GL_OES_egl_sync GL_OES_texture_npot GL_OES_surfaceless_context GL_EXT_discard_framebuffer GL_EXT_blend_minmax GL_EXT_multi_draw_arrays GL_EXT_multisampled_render_to_texture GL_EXT_shader_texture_lod GL_EXT_texture_format_BGRA8888 GL_EXT_occlusion_query_boolean GL_EXT_texture_rg GL_EXT_draw_buffers GL_EXT_shader_framebuffer_fetch GL_IMG_shader_binary GL_IMG_texture_compression_pvrtc GL_IMG_texture_compression_pvrtc2 GL_IMG_texture_npot GL_IMG_texture_format_BGRA8888 GL_IMG_read_format GL_IMG_program_binary GL_IMG_uniform_buffer_ob
[HOST] [HOST ] ect GL_IMG_multisampled_render_to_texture GL_KHR_debug
[HOST] [HOST ] 1453.734416 s: SGXFRMCPY: Create Done for resolution: 1920x1080 !!!
[HOST] [HOST ] 1453.734477 s: IPC_OUT_0 : Create in progress !!!
[HOST] [HOST ] 1453.734660 s: IPC_OUT_0 : Create Done !!!
[HOST] [IPU2 ] 48698.375068 s: IPC_IN_2 : Create in progress !!!
[HOST] [IPU2 ] 48698.376227 s: IPC_IN_2 : Create Done !!!
[HOST] [IPU2 ] 48698.376624 s: DISPLAY: Create in progress !!!
[HOST] [IPU2 ] 48698.376959 s: dispcore/src/vpscore_dss.c @ Line 465:
[HOST] [IPU2 ] 48698.377051 s: Display Controller registration failed for dss core instance 0
[HOST] [IPU2 ] 48698.377173 s: dispdrv/src/vpsdrv_displayCore.c @ Line 405:
[HOST] [IPU2 ] 48698.377264 s: Dss core open failed!!
[HOST] [IPU2 ] 48698.377325 s: dispdrv/src/vpsdrv_displayApi.c @ Line 326:
[HOST] [IPU2 ] 48698.377417 s: Core open failed!!
[HOST] [IPU2 ] 48698.377508 s: Assertion @ Line: 326 in displayLink_drv.c: pObj->displayHndl != NULL : failed !!!

over 8 years ago

0 Yordan Kamenov over 8 years ago

TI__Mastermind 42515 points

Hi,

which version is your VisonSDK?

Regards,
Yordan

0 Ian43425 over 8 years ago in reply to Yordan Kamenov

Prodigy 165 points

Sorry left that out... it is version 3.00.00

0 Yordan Kamenov over 8 years ago in reply to Ian43425

TI__Mastermind 42515 points

Hi,

I have forwarded your question to VisionSDk experts for comment.

Regards,
Yordan

0 Shiju Sivasankaran over 8 years ago in reply to Yordan Kamenov

TI__Genius 13395 points

What is the VSDK version you use?

If you suspect its due to the link ID related issyes, then Use "Auto usecase generation tool" for generating UC _priv.c and _priv.h files

BTW, object detection UC is not valiadted on HLOS side, and any RTOS side UC may not works as is from HLOS, some amount of porting is required.

regards, Shiju

0 Ian43425 over 8 years ago in reply to Shiju Sivasankaran

Prodigy 165 points

Hi Shiju

Thanks for the feedback.

It is version 3.00.00

I did use the Auto generation tool. The use-case successfully compiled but when run final Display stage fails to initialise (see the original console output I posted).

So it seems like I am quite close. But I am unsure how to proceed with debugging further, so any tips would be helpful.

For our application we require the use of linux (which as I understand the SDK means using the HLOS side). Are there any plans to validate this UC for HLOS? Can you tell me how complex the porting would be and indicate some pointers on where to start?

Ian

0 Shiju Sivasankaran over 8 years ago in reply to Ian43425

TI__Genius 13395 points

Hi Ian
As of today no plan to port this UC on VSDK Linux (HLOS).
you can debug the IPU side diaplay crash using CCS (code composer studio).
The UC porting on HLOS is mainly to port the OS dependent functionalities.

Regards, Shiju

0 Ian43425 over 8 years ago in reply to Shiju Sivasankaran

Prodigy 165 points

Hi Shiju

Thanks.

Regarding the porting
- is there any technical reason why this UC should *not* work on HLOS? ie if I chose to attempt the port - do I have a reasonable chance of success.
- using the existing SDK, do I have all the necessary source files to successfully port this UC?
- The other bit I do not understand is that the algorithms are executed on either the DSP, EVE or IPU with the results being passed to the A15 (whether it be HLOS or RTOS) via IPC. The IPC module appears to be very well defined and supported for both OS, So I would have thought the choice of HLOS or RTOS would not be material. Can you please expand a little on why my understanding here is incorrect?

Thanks for the assistance
Ian

0 Shiju Sivasankaran over 8 years ago in reply to Ian43425

TI__Genius 13395 points

Ian

can you please share the usecase.txt file with me, i would like to review?

regards, Shiju

0 Ian43425 over 8 years ago in reply to Shiju Sivasankaran

Prodigy 165 points

Hi Shiju

This is the usecase file that I cannot get the display working on

UseCase: chains_lvds_object_detect

Capture -> Alg_ImagePreProcess (EVE1) -> Dup_capture -> Merge_alg

Dup_capture -> Alg_ImgPyramid -> Dup_imgPyramid -> Alg_FPCompute (EVE1) -> Alg_ObjectDetection (DSP1) -> Merge_objectDetect (DSP1)

Dup_imgPyramid -> Merge_objectDetect (DSP1) -> Sync_objectDetect (DSP2) -> Alg_ObjectClassification (DSP2) -> Merge_alg -> Sync_alg -> Alg_ObjectDraw -> SgxFrmcpy (A15)

// display on m4
SgxFrmcpy -> Display_alg (IPU1_0)

GrpxSrc -> Display_Grpx

Since we last interacted - I have successfully created *another* usecase that does similar to the above but instead of displaying, it encodes the output to h264. I then took the h264 output and generated an RTSP/RTP stream so I could display the resulting h264 stream over the network on VLC.

This works, in that I can see road signs being read and pedestrians being detected. So my conclusion is that it seems the algorithms should work using hlos but I still cannot work out why the display won't work in my original usecase.

For interest that usecase for that is

UseCase: chains_lvds_object_h264_encode

Capture -> Alg_ImagePreProcess (EVE1) -> Dup_capture -> Merge_alg

Dup_capture -> Alg_ImgPyramid -> Dup_imgPyramid -> Alg_FPCompute (EVE1) -> Alg_ObjectDetection (DSP1) -> Merge_objectDetect (DSP1)

Dup_imgPyramid -> Dup_imgpmd_dsp (DSP2) -> Merge_objectDetect (DSP1) -> Sync_objectDetect (DSP2) -> Alg_ObjectClassification (DSP2) -> Merge_alg

// LD
Dup_imgpmd_dsp -> Alg_LaneDetect (DSP2) -> Merge_alg

Merge_alg -> Sync_alg -> Alg_ObjectDraw -> Encode

// display on m4
Encode -> Null (A15)

0 Shiju Sivasankaran over 8 years ago in reply to Ian43425

TI__Genius 13395 points

Ian
Nice to hear that the Algorithm flow works,
BTW, what is the display you use, is it an HDMI TV that support 1080p60?
FYI, We support only HDMI display with VSDK linux, no LCD/overlay etc. is supported.
Does the TV connected to the board? if not please connect and check again.

another suggestion, can you please remove SgxFrmcpy (A15) from the chain and try
// display on m4
Alg_ObjectDraw -> Display_alg (IPU1_0)

Regards, Shiju

0 Ian43425 over 8 years ago in reply to Shiju Sivasankaran

Prodigy 165 points

Hi Shiju

>> BTW, what is the display you use, is it an HDMI TV that support 1080p60?
The display is HDMI that supports 1080p60. The standard usecases (for example vip_single_cam_enc_dec_sgx_display) that are packages with the sdk work fine.

>> Does the TV connected to the board?
Yes it does

>> another suggestion, can you please remove SgxFrmcpy (A15) from the chain and try

Tried this and got the same error result

Full use case is

UseCase: chains_lvds_object_detect

Capture -> Alg_ImagePreProcess (EVE1) -> Dup_capture -> Merge_alg

Dup_capture -> Alg_ImgPyramid -> Dup_imgPyramid -> Alg_FPCompute (EVE1) -> Alg_ObjectDetection (DSP1) -> Merge_objectDetect (DSP1)

Dup_imgPyramid -> Merge_objectDetect (DSP1) -> Sync_objectDetect (DSP2) -> Alg_ObjectClassification (DSP2) -> Merge_alg -> Sync_alg -> Alg_ObjectDraw -> Display_alg (IPU1_0)

GrpxSrc -> Display_Grpx

0 Shiju Sivasankaran over 8 years ago in reply to Ian43425

TI__Genius 13395 points

Ian
Thanks for the details.
can you please share the dsiply link confiuration paramters with me?
for both Display_alg & Display_Grpx

regards, Shiju

Processors

Processors forum

Linux/TDA2EVM5777: Display controller registration fails with object detection usecase