• State Resolved
  • Locked Locked
  • Replies 5 replies
  • Subscribers 99 subscribers
  • Views 285 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

J721EXSOMXEVM: Vehicle detection based on avp4

yang yang
Prodigy 225 points
Part Number: J721EXSOMXEVM

Hi

I want to use cameras for vehicle detection, so I delete the  post process in AVP4, then add vd's post_proc_moduleand draw_detections_module in AVP to AVP4, and then replacing the network and config of TIDL.

The images can be displayed normally，but don't display the detection box of vehicle. I pressed 'P', it displays as follows. Can you provide some advice?

RTOS: 08-04.

Summary of CPU load,
====================

CPU: mpu1_0: TOTAL LOAD =   1.70 % ( HWI =   0.56 %, SWI =   0. 0 % )
CPU: mcu1_0: TOTAL LOAD =   1. 0 % ( HWI =   0. 0 %, SWI =   0. 0 % )
CPU: mcu2_0: TOTAL LOAD =  19. 0 % ( HWI =   0. 0 %, SWI =   0. 0 % )
CPU: mcu2_1: TOTAL LOAD =   1. 0 % ( HWI =   0. 0 %, SWI =   0. 0 % )
CPU: mcu3_0: TOTAL LOAD =   1. 0 % ( HWI =   0. 0 %, SWI =   0. 0 % )
CPU:  c6x_1: TOTAL LOAD =   8. 0 % ( HWI =   0. 0 %, SWI =   0. 0 % )
CPU:  c6x_2: TOTAL LOAD =   1. 0 % ( HWI =   0. 0 %, SWI =   0. 0 % )
CPU:  c7x_1: TOTAL LOAD =  27. 0 % ( HWI =   0. 0 %, SWI =   0. 0 % )


HWA performance statistics,
===========================

HWA:   LDC : LOAD =  21.67 % ( 130 MP/s )
HWA:   MSC0: LOAD =  32.98 % ( 198 MP/s )
HWA:   MSC1: LOAD =  11. 7 % ( 71 MP/s )


DDR performance statistics,
===========================

DDR: READ  BW: AVG =   1804 MB/s, PEAK =  13824 MB/s
DDR: WRITE BW: AVG =    725 MB/s, PEAK =   6348 MB/s
DDR: TOTAL BW: AVG =   2529 MB/s, PEAK =  20172 MB/s


Detailed CPU performance/memory statistics,
===========================================

DDR_SHARED_MEM: Alloc's: 100 alloc's of 108348444 bytes
DDR_SHARED_MEM: Free's : 3 free's  of 279296 bytes
DDR_SHARED_MEM: Open's : 97 allocs  of 108069148 bytes
DDR_SHARED_MEM: Total size: 536870912 bytes

CPU: mcu1_0: TASK:           IPC_RX:   0. 0 %
CPU: mcu1_0: TASK:       REMOTE_SRV:   0.32 %
CPU: mcu1_0: TASK:        LOAD_TEST:   0. 0 %
CPU: mcu1_0: TASK:      IPC_TEST_RX:   0. 0 %
CPU: mcu1_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu1_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu1_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu1_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu1_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu1_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu1_0: TASK:      IPC_TEST_TX:   0. 0 %

CPU: mcu1_0: HEAP:    DDR_LOCAL_MEM: size =    8388608 B, free =    8388352 B ( 99 % unused)

CPU: mcu2_0: TASK:           IPC_RX:   0.75 %
CPU: mcu2_0: TASK:       REMOTE_SRV:   0.70 %
CPU: mcu2_0: TASK:        LOAD_TEST:   0. 0 %
CPU: mcu2_0: TASK:       TIVX_CPU_0:   0. 0 %
CPU: mcu2_0: TASK:      TIVX_VPAC_N:   0. 0 %
CPU: mcu2_0: TASK:      TIVX_VPAC_L:   2.20 %
CPU: mcu2_0: TASK:      TIVX_VPAC_M:   5. 6 %
CPU: mcu2_0: TASK:      TIVX_VPAC_M:   0. 0 %
CPU: mcu2_0: TASK:      TIVX_VPAC_V:   0. 0 %
CPU: mcu2_0: TASK:       TIVX_CAPT1:   1.15 %
CPU: mcu2_0: TASK:       TIVX_CAPT2:   0. 0 %
CPU: mcu2_0: TASK:       TIVX_DISP1:   0.84 %
CPU: mcu2_0: TASK:       TIVX_DISP2:   0. 0 %
CPU: mcu2_0: TASK:       TIVX_CSITX:   0. 0 %
CPU: mcu2_0: TASK:       TIVX_CAPT3:   0. 0 %
CPU: mcu2_0: TASK:       TIVX_CAPT4:   0. 0 %
CPU: mcu2_0: TASK:       TIVX_CAPT5:   0. 0 %
CPU: mcu2_0: TASK:       TIVX_CAPT6:   0. 0 %
CPU: mcu2_0: TASK:       TIVX_CAPT7:   0. 0 %
CPU: mcu2_0: TASK:       TIVX_CAPT8:   0. 0 %
CPU: mcu2_0: TASK:      TIVX_DISP_M:   0. 0 %
CPU: mcu2_0: TASK:      TIVX_DISP_M:   0. 0 %
CPU: mcu2_0: TASK:      TIVX_DISP_M:   0. 0 %
CPU: mcu2_0: TASK:      TIVX_DISP_M:   0. 0 %

CPU: mcu2_0: HEAP:    DDR_LOCAL_MEM: size =   16777216 B, free =   16693248 B ( 99 % unused)
CPU: mcu2_0: HEAP:           L3_MEM: size =     262144 B, free =     261888 B ( 99 % unused)

CPU: mcu2_1: TASK:           IPC_RX:   0. 0 %
CPU: mcu2_1: TASK:       REMOTE_SRV:   0.28 %
CPU: mcu2_1: TASK:        LOAD_TEST:   0. 0 %
CPU: mcu2_1: TASK:         TIVX_SDE:   0. 0 %
CPU: mcu2_1: TASK:         TIVX_DOF:   0. 0 %
CPU: mcu2_1: TASK:       TIVX_CPU_1:   0. 0 %
CPU: mcu2_1: TASK:      IPC_TEST_RX:   0. 0 %
CPU: mcu2_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu2_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu2_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu2_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu2_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu2_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu2_1: TASK:      IPC_TEST_TX:   0. 0 %

CPU: mcu2_1: HEAP:    DDR_LOCAL_MEM: size =   16777216 B, free =   16773376 B ( 99 % unused)
CPU: mcu2_1: HEAP:           L3_MEM: size =     262144 B, free =     262144 B (100 % unused)

CPU: mcu3_0: TASK:           IPC_RX:   0. 0 %
CPU: mcu3_0: TASK:       REMOTE_SRV:   0.22 %
CPU: mcu3_0: TASK:        LOAD_TEST:   0. 0 %
CPU: mcu3_0: TASK:      TIVX_MCU3_0:   0. 0 %
CPU: mcu3_0: TASK:      IPC_TEST_RX:   0. 0 %
CPU: mcu3_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu3_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu3_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu3_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu3_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu3_0: TASK:      IPC_TEST_TX:   0. 0 %
CPU: mcu3_0: TASK:      IPC_TEST_TX:   0. 0 %

CPU: mcu3_0: HEAP:    DDR_LOCAL_MEM: size =    8388608 B, free =    8388352 B ( 99 % unused)

CPU:  c6x_1: TASK:           IPC_RX:   0. 4 %
CPU:  c6x_1: TASK:       REMOTE_SRV:   0. 7 %
CPU:  c6x_1: TASK:        LOAD_TEST:   0. 0 %
CPU:  c6x_1: TASK:         TIVX_CPU:   7.92 %
CPU:  c6x_1: TASK:      IPC_TEST_RX:   0. 0 %
CPU:  c6x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_1: TASK:      IPC_TEST_TX:   0. 0 %

CPU:  c6x_1: HEAP:    DDR_LOCAL_MEM: size =   16777216 B, free =   16753408 B ( 99 % unused)
CPU:  c6x_1: HEAP:           L2_MEM: size =     229376 B, free =          0 B (  0 % unused)
CPU:  c6x_1: HEAP:  DDR_SCRATCH_MEM: size =   50331648 B, free =   50331648 B (100 % unused)

CPU:  c6x_2: TASK:           IPC_RX:   0. 1 %
CPU:  c6x_2: TASK:       REMOTE_SRV:   0. 6 %
CPU:  c6x_2: TASK:        LOAD_TEST:   0. 0 %
CPU:  c6x_2: TASK:         TIVX_CPU:   0.51 %
CPU:  c6x_2: TASK:      IPC_TEST_RX:   0. 0 %
CPU:  c6x_2: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_2: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_2: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_2: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_2: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_2: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c6x_2: TASK:      IPC_TEST_TX:   0. 0 %

CPU:  c6x_2: HEAP:    DDR_LOCAL_MEM: size =   16777216 B, free =   16540416 B ( 98 % unused)
CPU:  c6x_2: HEAP:           L2_MEM: size =     229376 B, free =     229376 B (100 % unused)
CPU:  c6x_2: HEAP:  DDR_SCRATCH_MEM: size =   50331648 B, free =   50331648 B (100 % unused)

CPU:  c7x_1: TASK:           IPC_RX:   0. 3 %
CPU:  c7x_1: TASK:       REMOTE_SRV:   0. 7 %
CPU:  c7x_1: TASK:        LOAD_TEST:   0. 0 %
CPU:  c7x_1: TASK:      TIVX_C7_1_P:  26.13 %
CPU:  c7x_1: TASK:      TIVX_C7_1_P:   0. 0 %
CPU:  c7x_1: TASK:      TIVX_C7_1_P:   0. 0 %
CPU:  c7x_1: TASK:      TIVX_C7_1_P:   0. 0 %
CPU:  c7x_1: TASK:      TIVX_C7_1_P:   0. 0 %
CPU:  c7x_1: TASK:      TIVX_C7_1_P:   0. 0 %
CPU:  c7x_1: TASK:      TIVX_C7_1_P:   0. 0 %
CPU:  c7x_1: TASK:      TIVX_C7_1_P:   0. 0 %
CPU:  c7x_1: TASK:      IPC_TEST_RX:   0. 0 %
CPU:  c7x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c7x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c7x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c7x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c7x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c7x_1: TASK:      IPC_TEST_TX:   0. 0 %
CPU:  c7x_1: TASK:      IPC_TEST_TX:   0. 0 %

CPU:  c7x_1: HEAP:    DDR_LOCAL_MEM: size =  268435456 B, free =  206148352 B ( 76 % unused)
CPU:  c7x_1: HEAP:           L3_MEM: size =    8159232 B, free =          0 B (  0 % unused)
CPU:  c7x_1: HEAP:           L2_MEM: size =     458752 B, free =     458752 B (100 % unused)
CPU:  c7x_1: HEAP:           L1_MEM: size =      16384 B, free =          0 B (  0 % unused)
CPU:  c7x_1: HEAP:  DDR_SCRATCH_MEM: size =  385875968 B, free =  382213397 B ( 99 % unused)


GRAPH: app_tidl_avp5_graph (#nodes =   9, #executions =   6373)
 NODE:       CAPTURE1:             capture_node: avg =  33239 usecs, min/max =  32536 /  45093 usecs, #executions =       6373
 NODE:      VPAC_LDC1:                 ldc_node: avg =   7047 usecs, min/max =   6855 /   8027 usecs, #executions =       6373
 NODE:      VPAC_MSC1:              scaler_node: avg =  10450 usecs, min/max =  10334 /  11126 usecs, #executions =       6373
 NODE:          DSP-1:            ODPreProcNode: avg =   1784 usecs, min/max =   1774 /   2163 usecs, #executions =       6373
 NODE:       DSP_C7-1:               ODTIDLNode: avg =   8676 usecs, min/max =   8643 /   8940 usecs, #executions =       6373
 NODE:          DSP-2:           ODPostProcNode: avg =    367 usecs, min/max =    105 /   4006 usecs, #executions =       6373
 NODE:          DSP-1:       DrawDetectionsNode: avg =   3571 usecs, min/max =   3544 /   4229 usecs, #executions =       6373
 NODE:      VPAC_MSC1:              mosaic_node: avg =   4090 usecs, min/max =   3724 /  32790 usecs, #executions =       6373
 NODE:       DISPLAY1:              DisplayNode: avg =   7503 usecs, min/max =     99 /  16573 usecs, #executions =       6373

 PERF:           FILEIO: avg =      0 usecs, min/max = 4294967295 /      0 usecs, #executions =          0
 PERF:            TOTAL: avg =  41665 usecs, min/max =  40827 /  42546 usecs, #executions =         21

 PERF:            TOTAL:   24. 0 FPS

  • 0
    TI__Mastermind 24041 points
    yang yang said:
    I want to use cameras for vehicle detection, so I delete the  post process in AVP4, then add vd's post_proc_moduleand draw_detections_module in AVP to AVP4, and then replacing the network and config of TIDL.

    Not complete sure what changes were exactly added.

    Please list out the file names that you have deleted along with paths, followed by detailed explanation on post processing changes.

    Since this is custom change you have added in OpneVx based demo, can you validate the post processing node is coded correctly ? (Accepting the bounding box co ordinates, processing them them correctly ? If its yes how are you making sure that those code changes are suitable for AVP4 parking ?

    yang yang said:
    I delete the  post process in AVP4, then add vd's post_proc_moduleand draw_detections_module in AVP to AVP4,

    What do you mean by vd's  here ?

  • 0 in reply to Pratik Kedar
    Prodigy 225 points

    VD means vehicle detection, PSD means parking spot detection.

    In avp4, i deleted app_post_proc_module.c in ti-processor-sdk-rtos-j721e-evm-08_04_00_06/vision_apps/apps/dl_demos/app_tidl_avp4 ，the camera outputs yuv422, so i disabled viss, aweb, dof, hist, srv.

    I noticed that the data flow of AVP is shown in the following figure.

    So i add avp_post_proc_module.c, avp_draw_detections_module.c and fisheyes_angle_table.c in ti-processor-sdk-rtos-j721e-evm-08_04_00_06/vision_apps/apps/dl_demos/app_tidl_avp to avp4.

    About post_processing, the functins like this, i just deleted the functions xx_xx_xx_pc(related to semantic segmentation), and replaced NUM_CH by cameras enable.

    The default and updated post-processing parameters are as follows:

    static void set_post_proc_defaults_od(PostProcObj *postProcObj)
{
    /*Focal length is hard coded for the given camera property. It can be exposed by cfg*/
    postProcObj->params.num_max_det = 100;
    postProcObj->params.points_per_line = 64;
    postProcObj->params.num_keypoints = 4;
    postProcObj->params.focal_length = 311.833f;
    postProcObj->params.inter_center_x_fact = 10;
    postProcObj->params.inter_center_y_fact = 2;
    postProcObj->params.num_table_rows = ANGLE_TABLE_ROWS;
    postProcObj->params.output_buffer_offset = 0;
}

static void update_post_proc_params_od(AppObj *obj, PostProcObj *postProcObj)
{
    postProcObj->params.width     = obj->scalerObj.output[0].width;
    postProcObj->params.height    = obj->scalerObj.output[0].height;
    postProcObj->params.dl_width  = obj->scalerObj.output[0].width;
    postProcObj->params.dl_height = obj->scalerObj.output[0].height;
    postProcObj->params.center_x  = postProcObj->params.width/2;
    postProcObj->params.center_y  = postProcObj->params.height/2;
}

    In the cfg file,

    Tidl_config: /opt/vision_ app/test_ data/psdkra/tidl_ models/tidl_ io_ jvdNet_ 1. bin

    Tidl_network: /opt/vision_ app/test_ data/psdkra/tidl_ models/tidl_ net_ jvdNet. bin

    Now, i can see the green boxes. Indoors, it seems to be able to detect many rectangular objects, but the position of the green box is not accurate, and this is also the case when detecting vehicles outdoors.

    Are these incorrect boundary coordinate boxes mainly caused by post-processing?

  • 0 in reply to yang yang
    TI__Guru* 85266 points

    Hi,

    To understand your usecase currently, 

    You are using the AVP4 demo, but you are not doing segmentation, but you wanted to do vehicle detection instead and your input source is camera (YUV422) right?

    You are using the same model, i.e. tidl_net_jvdNet.bin and tidl_ io_ jvdNet_ 1. bin, that was used in the avp demo for vehicle detection right?

    In this case, your current graph should be :

    Capture -> LDC -> MSC -> Pre-Proc -> TIDL node -> Post-proc -> DrawDet -> Mosaic -> Display

    Is my understanding correct?

    I see that the input size of the model is 512 x 512 for vd, whereas it is 768 x 384 for seg. Have you taken care of this in the cfg file of the avp4 demo?

    Could you share your updated config file?

    Regards,

    Nikhil

  • 0 in reply to Nikhil Dasan
    TI__Guru* 85266 points

    Hi,

    It seems that your response got somehow deleted, but I had a backup in my mail... 

    Pasting the same here

    "Yes, your understanding is correct.

    In config file, i added enable_drawdet to control DrawDet.  Tidl input size  is 768 x 384. I noticed the model is 768 x 384 for vd in 'ti-processor-sdk-rtos-j721e-evm-08_04_00_06/vision_apps/apps/dl_demos/app_tidl_avp2(the demo also can run vehicle detection)', so i think vd model can accept the size."

    # enable or disable Capture
enable_capture 1

# enable or disable VISS
enable_viss 0

# enable or disable AEWB
enable_aewb 0

# enable or disable LDC
enable_ldc 1

# enable or disable Scaler
enable_scaler 1

# enable or disable DOF
enable_dof 0

# enable or disable TIDL Pre Proc
enable_pre_proc 1

# enable or disable TIDL
enable_tidl 1

# enable or disable TIDL Post Proc
enable_post_proc 1

# enable or disable SRV
enable_srv 0

# enable or disable HIST
enable_hist 0

# enable or disable SW mosaic
enable_mosaic 1

# enable or disable Display
enable_display 1

# enable or disable Pipeline mode of execution
pipeline_exec 1

# Maximum number of cameras enabled
num_cameras_enabled 4

# enable or disable draw detections
enable_drawdet  1

# location of config
tidl_config   /opt/vision_apps/test_data/psdkra/tidl_models/tidl_io_jvdNet_1.bin

# location of network
tidl_network  /opt/vision_apps/test_data/psdkra/tidl_models/tidl_net_jvdNet.bin

# size given to DL network (width height)
# This should should not be less than 4x of input width or height
dl_size 768 384

# enable interactive mode 0-noninteractive, 1-interactive
is_interactive    1

# If 1 - Enable display 0 - Disable display
display_option      1

# number of iterations to loop the inputs
num_iterations      1000000


viz_th 0.9

# number of classes for each output tensor for visualization. For each output tensor, number of clases to be provided with spaces
# Semantic Segmentation, Motion, Depth
num_classes 5 2 255

# Enable or disable output image writing. 1 Enables it , 0 disables it
en_out_img_write    0

# Input can be provided as either 0-RGB or 1-YUV (default)
ip_rgb_or_yuv       1

# Output can be provided as either 0-RGB or 1-YUV (default)
op_rgb_or_yuv       1






# Bowl offset
offsetXleft    -400
offsetXright   400
offsetYfront   -450
offsetYback    450

# Num views
num_views 10

# view 0 parameters
camx 0
camy 0
camz 200
targetx 0
targety 0
targetz 0
anglex 0
angley 0
anglez 0

# view 1 parameters
camx 0
camy 0
camz 400
targetx 0
targety 0
targetz 0
anglex 0
angley 0
anglez 0

# view 2 parameters
camx 0
camy -260
camz 240
targetx 0
targety 0
targetz 0
anglex 0
angley 0
anglez 0

# view 3 parameters
camx 0
camy -260
camz 160
targetx -5
targety 0
targetz 0
anglex 0
angley 0
anglez 0.5

# view 4 parameters
camx 0
camy -260
camz 160
targetx -5
targety 0
targetz 0
anglex 0
angley 0
anglez 1.5

# view 5 parameters
camx 0
camy -260
camz 160
targetx -5
targety 0
targetz 0
anglex 0
angley 0
anglez 2.50

# view 6 parameters
camx 0
camy -280
camz 160
targetx -10
targety 0
targetz 0
anglex 0
angley 0
anglez 3.14

# view 7 parameters
camx 0
camy -280
camz 160
targetx -10
targety 0
targetz 0
anglex 0
angley 0
anglez 4.0

# view 8 parameters
camx 0
camy -280
camz 160
targetx -10
targety 0
targetz 0
anglex 0
angley 0
anglez 5.0

# view 9 parameters
camx 0
camy -280
camz 160
targetx -10
targety 0
targetz 0
anglex 0
angley 0
anglez 5.5

  • +1 in reply to Nikhil Dasan
    TI__Guru* 85266 points

    Hi,

    The model used in avp2 is onnx_tiad_ssd_1.bin whereas jvdNet is used in avp.

    Could you try giving the input size as 512 x 512 and check if this resolves the issue?

    Regards,

    Nikhil