This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM67A: Compiling models for specific TIDL

Part Number: AM67A

Hi,
I’m new to the EdgeAI workflow and I’m trying to compile an existing depth_estimation model from the TI Model Zoo for a specific TIDL version 11_00_06_00 targeting AM67A. I’m running into a few issues and would appreciate some guidance.

What I’m trying to do

I added the following custom model entry in
edgeai-tidl-tools/examples/osrt_python/model_configs.py:

'de-7300' : create_model_config(
    task_type="depth_estimation",
    source=dict(
        model_url="">software-dl.ti.com/.../fast-depth.onnx",
        infer_shape=True,
    ),
    preprocess=dict(
        resize=256,
        crop=224,
        data_layout="NCHW",
        resize_with_pad=False,
        reverse_channels=False,
    ),
    session=dict(
        session_name="onnxrt",
        model_path=os.path.join(models_base_path , "fast-depth.onnx"),
        input_mean=[123.675, 116.28, 103.53],
        input_scale=[0.017125, 0.017507, 0.017429],
        input_optimization=True,
    ),
    postprocess=dict(),
    extra_info=dict(
        num_images = numImages ,
        num_classes = 1000
    ),
    runtime_options = {
        'advanced_options:inference_mode' : 2,
        'advanced_options:num_cores' : 1,
        'advanced_options:calibration_frames' : 12,
        'advanced_options:calibration_iterations' : 12
    }
)

 

Issue

The compilation completes, but inside
model-artifacts/de-7300/artifacts/
no .bin files are generated. Instead, I only get a tempDir that contains temporary .bin files.

When I try to run those temporary bin files, they fail at runtime.

Additional problem

I also tried using edgeai-benchmark, but I couldn’t find a way to set the required TIDL version 11_00_06_00, which leads to version-mismatch errors.

Questions

  1. What is the correct procedure to compile a Model Zoo model for a specific TIDL version (11_00_06_00)?

  2. Is edgeai-benchmark expected to support manual TIDL version selection, or should it match the internal version of the tools repository?

Any guidance or examples would be very helpful.
Thank you!

  • Hi Andow,

    I did not see what was generated, but if onnxrt_ep.py finsihed OK the .bin files in model-artifacts/<model>tempDir will be the runnable artifacts.  Copy the entire model-artifacts onto the EVM under /home/root/edgeai-tidl-tools.  Assuming the versions match, you can run them with onnxrt_ep.py on the EVM.

    1. In the edgeai-modelzoo repository, checkout the branch of the version you want.

    2. I do not totally understand your question but I think the answer is it should match the repository (there will be tags/branches for your version)  Don't try to run stuff for a different version 

    Regards,

    Chris 

  • Thanks for the response. Here is the logging — it seems I managed to configure everything correctly, and the model was generated properly. But to test it, I wanted to use the repository https://github.com/TexasInstruments-Sandbox/edgeai-demo-monodepth-estimation.

    When I plug my model into run_demo.sh, I get the following log:

    root@j722s-evm:/opt/edgeai-demo-monodepth-estimation# ./run_demo.sh 
    [DEBUG] Loading model: /opt/model_zoo/de-7300_onnxrt_fast-depth/model/fast-depth.onnx
    [DEBUG] File exists: True
    [DEBUG] Providers: ['TIDLExecutionProvider', 'CPUExecutionProvider']
    [DEBUG] Provider options: [{'artifacts_folder': '/opt/model_zoo/de-7300_onnxrt_fast-depth/artifacts', 'debug_level': 0, 'advanced_options:output_feature_16bit_names_list': '511, 983 '}, {}]
    libtidl_onnxrt_EP loaded 0x2efdafa0 
    Final number of subgraphs created are : 1, - Offloaded Nodes - 84, Total Nodes - 84 
    APP: Init ... !!!
      1466.410979 s: MEM: Init ... !!!
      1466.411078 s: MEM: Initialized DMA HEAP (fd=5) !!!
      1466.411305 s: MEM: Init ... Done !!!
      1466.411347 s: IPC: Init ... !!!
      1466.467609 s: IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
      1466.474972 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
      1466.475172 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
      1466.475193 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
      1466.475202 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
      1466.476295 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 
      1466.476656 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 
      1466.477113 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 
      1466.477455 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 
      1466.477503 s:  VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!!
      1466.477549 s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
    [DEBUG] InferenceSession created successfully ✅
    Calculating output tensor dimensions and offsets...
    model datatype : float32
    caps: video/x-raw, width=1920, height=1080, format=RGB, framerate=0/1
    Parsing GST pipeline: 
    input: v4l2src device=/dev/video-usb-cam0  ! image/jpeg,width=1024,height=768 ! jpegdec ! tiovxdlcolorconvert  !  video/x-raw, format=NV12 ! queue leaky=2 max-size-buffers=2  ! tiovxmultiscaler name=split_resize     split_resize.   ! video/x-raw, width=328, height=246, format=NV12  ! tiovxmultiscaler target=1 ! queue leaky=2 max-size-buffers=2 ! video/x-raw, width=246, height=246  ! tiovxdlpreproc out-pool-size=4 data-type=float32   channel-order=nchw tensor-format=rgb  mean-0=0.000000 mean-1=0.000000 mean-2=0.000000 scale-0=0.003922 scale-1=0.003922 scale-2=0.003922  ! application/x-tensor-tiovx  ! appsink name=tensor_in max-buffers=2 drop=True    split_resize. ! queue leaky=2 max-size-buffers=2  ! video/x-raw, width=1536, height=864, format=NV12 ! tiovxdlcolorconvert out-pool-size=4 ! video/x-raw, format=RGB ! appsink name=image_in max-buffers=2 drop=True
    
    output:  appsrc format=GST_FORMAT_TIME is-live=true  name=out ! video/x-raw,  format=RGB, width=1920, height=1080  ! queue ! tiovxdlcolorconvert out-pool-size=4  ! video/x-raw, format=NV12  !  tiperfoverlay main-title=""  ! kmssink sync=false driver-name=tidss  plane-id=31 force-modesetting=True
    
    Starting with in_gst: 
    v4l2src device=/dev/video-usb-cam0  ! image/jpeg,width=1024,height=768 ! jpegdec ! tiovxdlcolorconvert  !  video/x-raw, format=NV12 ! queue leaky=2 max-size-buffers=2  ! tiovxmultiscaler name=split_resize     split_resize.   ! video/x-raw, width=328, height=246, format=NV12  ! tiovxmultiscaler target=1 ! queue leaky=2 max-size-buffers=2 ! video/x-raw, width=246, height=246  ! tiovxdlpreproc out-pool-size=4 data-type=float32   channel-order=nchw tensor-format=rgb  mean-0=0.000000 mean-1=0.000000 mean-2=0.000000 scale-0=0.003922 scale-1=0.003922 scale-2=0.003922  ! application/x-tensor-tiovx  ! appsink name=tensor_in max-buffers=2 drop=True    split_resize. ! queue leaky=2 max-size-buffers=2  ! video/x-raw, width=1536, height=864, format=NV12 ! tiovxdlcolorconvert out-pool-size=4 ! video/x-raw, format=RGB ! appsink name=image_in max-buffers=2 drop=True
    
    
    out gst:  appsrc format=GST_FORMAT_TIME is-live=true  name=out ! video/x-raw,  format=RGB, width=1920, height=1080  ! queue ! tiovxdlcolorconvert out-pool-size=4  ! video/x-raw, format=NV12  !  tiperfoverlay main-title=""  ! kmssink sync=false driver-name=tidss  plane-id=31 force-modesetting=True
    Starting GST pipeline
    pull buffers
    pull buffers
    pull buffers
    ...


    and then:

    [ 1459.828981] usb 1-1.3: reset high-speed USB device number 5 using xhci-hcd
    [ 1466.932964] python3 invoked oom-killer: gfp_mask=0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), order=0, oom_score_adj=0
    [ 1466.952180] CPU: 3 UID: 0 PID: 1639 Comm: python3 Not tainted 6.12.33-g1df32b6e7c23 #1
    [ 1466.952200] Hardware name: BeagleBoard.org BeagleY-AI (DT)
    [ 1466.952205] Call trace:
    [ 1466.952207]  dump_backtrace+0x90/0xe8
    [ 1466.952227]  show_stack+0x18/0x24
    [ 1466.952236]  dump_stack_lvl+0x74/0x8c
    [ 1466.952244]  dump_stack+0x18/0x24
    [ 1466.952249]  dump_header+0x3c/0x1a0
    [ 1466.952261]  oom_kill_process+0x130/0x348
    [ 1466.952269]  out_of_memory+0xdc/0x344
    [ 1466.952277]  __alloc_pages_noprof+0xc1c/0xcf0
    [ 1466.952285]  __folio_alloc_noprof+0x14/0x60
    [ 1466.952291]  __filemap_get_folio+0x21c/0x3ac
    [ 1466.952299]  filemap_fault+0x114/0xbb4
    [ 1466.952307]  __do_fault+0x3c/0x218
    [ 1466.952316]  __handle_mm_fault+0x8f8/0x12d0
    [ 1466.952324]  handle_mm_fault+0x15c/0x27c
    [ 1466.952332]  do_page_fault+0x110/0x4f8
    [ 1466.952339]  do_translation_fault+0xa8/0xb8
    [ 1466.952345]  do_mem_abort+0x44/0x94
    [ 1466.952352]  el0_ia+0x60/0xf8
    [ 1466.952358]  el0t_64_sync_handler+0xcc/0x12c
    [ 1466.952364]  el0t_64_sync+0x190/0x194
    [ 1466.952372] Mem-Info:
    [ 1467.082823] active_anon:174 inactive_anon:614962 isolated_anon:0
    [ 1467.082823]  active_file:1 inactive_file:0 isolated_file:0
    [ 1467.082823]  unevictable:0 dirty:0 writeback:0
    [ 1467.082823]  slab_reclaimable:3512 slab_unreclaimable:6192
    [ 1467.082823]  mapped:348 shmem:2446 pagetables:2318
    [ 1467.082823]  sec_pagetables:0 bounce:0
    [ 1467.082823]  kernel_misc_reclaimable:0
    [ 1467.082823]  free:12171 free_pcp:1 free_cma:818
    [ 1467.161673] Node 0 active_anon:696kB inactive_anon:2459848kB active_file:156kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1356kB dirty:0kB writeback:0kB shmem:9784kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:1683456kB writeback_tmp:0kB kernel_stack:3216kB pagetables:9272kB sec_pagetables:0kB all_unreclaimable? no
    [ 1467.192749] DMA free:31912kB boost:0kB min:28128kB low:35160kB high:42192kB reserved_highatomic:0KB active_anon:0kB inactive_anon:1533240kB active_file:40kB inactive_file:16kB unevictable:0kB writepending:0kB present:2097152kB managed:1618088kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
    [ 1467.224664] lowmem_reserve[]: 0 0 949 949
    [ 1467.234627] Normal free:16880kB boost:0kB min:16924kB low:21152kB high:25380kB reserved_highatomic:0KB active_anon:696kB inactive_anon:926608kB active_file:200kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:971992kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:3380kB
    [ 1467.270832] lowmem_reserve[]: 0 0 0 0
    [ 1467.274524] DMA: 1038*4kB (UME) 568*8kB (UME) 307*16kB (UME) 146*32kB (UME) 57*64kB (UME) 38*128kB (UME) 14*256kB (UME) 3*512kB (M) 0*1024kB 0*2048kB 0*4096kB = 31912kB
    [ 1467.301534] Normal: 415*4kB (UEC) 196*8kB (UEC) 117*16kB (UEC) 74*32kB (UEC) 35*64kB (UEC) 14*128kB (UE) 4*256kB (EC) 3*512kB (EC) 3*1024kB (UC) 0*2048kB 0*4096kB = 17132kB
    [ 1467.325234] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
    [ 1467.342869] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=32768kB
    [ 1467.354879] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
    [ 1467.376882] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=64kB
    [ 1467.389695] 2505 total pagecache pages
    [ 1467.393516] 0 pages in swap cache
    [ 1467.397362] Free swap  = 0kB
    [ 1467.404126] Total swap = 0kB
    [ 1467.410870] 1048576 pages RAM
    [ 1467.413850] 0 pages HighMem/MovableOnly
    [ 1467.425217] 401056 pages reserved
    [ 1467.433608] 229376 pages cma reserved
    [ 1467.441356] 0 pages hwpoisoned
    [ 1467.447299] Tasks state (memory values in pages):
    [ 1467.456604] [  pid  ]   uid  tgid total_vm      rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name
    [ 1467.478510] [    135]   998   135     1163       32       32        0         0    45056        0             0 rpcbind
    [ 1467.498891] [    136]     0   136     9982      512      256        0       256    77824        0          -250 systemd-journal
    [ 1467.519838] [    174]     0   174     7475      756      736       20         0    81920        0         -1000 systemd-udevd
    [ 1467.532217] [    182]   992   182     3827      192      192        0         0    65536        0             0 systemd-resolve
    [ 1467.552994] [    184]   991   184    22282      192      192        0         0    77824        0             0 systemd-timesyn
    [ 1467.573869] [    186]     0   186     3712      224      192       32         0    69632        0             0 systemd-userdbd
    [ 1467.595886] [    283]   999   283     1714       99       87       12         0    49152        0          -900 dbus-broker-lau
    [ 1467.617450] [    349]   999   349      830       57       57        0         0    40960        0          -900 dbus-broker
    [ 1467.638202] [    350]     0   350      620       44       32       12         0    40960        0             0 atd
    [ 1467.659034] [    355]   995   355     1855       82       64       18         0    53248        0             0 avahi-daemon
    [ 1467.672234] [    357]     0   357      725       47       32       15         0    40960        0             0 crond
    [ 1467.685156] [    361]     0   361    19680       43       32       11         0    49152        0             0 irqbalance
    [ 1467.706081] [    362]     0   362      575        0        0        0         0    40960        0             0 mstpd
    [ 1467.726949] [    363]     0   363     2470      114       96       18         0    53248        0             0 ofonod
    [ 1467.746960] [    364]   990   364    57592     1376     1361       15         0    94208        0             0 pulseaudio
    [ 1467.766959] [    366]     0   366      699       32       32        0         0    40960        0             0 starter
    [ 1467.786778] [    370]     0   370     3798      192      192        0         0    65536        0             0 systemd-logind
    [ 1467.809685] [    380]     0   380      723        7        0        7         0    40960        0             0 tee-supplicant
    [ 1467.821410] [    395]     0   395      875       11        0       11         0    45056        0             0 telnetd
    [ 1467.832275] [    397]   993   397     4153      221      192       29         0    65536        0             0 systemd-network
    [ 1467.843793] [    398]   995   398     1807       69       69        0         0    53248        0             0 avahi-daemon
    [ 1467.855074] [    432]     0   432   299766      416      416        0         0   208896        0             0 charon
    [ 1467.865842] [    544]     0   544     4160      288      288        0         0    73728        0             0 wpa_supplicant
    [ 1467.877295] [    551]     0   551      846       42       32       10         0    45056        0             0 lldpd
    [ 1467.887961] [    552]     0   552      775        4        0        4         0    40960        0             0 netserver
    [ 1467.916747] [    555]     0   555     3585      800      800        0         0    65536        0             0 snmpd
    [ 1467.951008] [    559]     0   559   466109     3885     3885        0         0   274432        0          -999 containerd
    [ 1467.994596] [    563]   997   563      819       33       33        0         0    45056        0             0 lldpd
    [ 1468.047301] [    566]     0   566      566        7        0        7         0    40960        0             0 agetty
    [ 1468.105609] [   1292]     0  1292      743       43       32       11         0    40960        0             0 dropbear
    [ 1468.121433] [   1293]     0  1293     1454      579      544       35         0    45056        0             0 sh
    [ 1468.136099] [   1425]     0  1425     1656      122       96       26         0    53248        0             0 login
    [ 1468.208224] [   1431]     0  1431     4357      360      352        8         0    73728        0           100 systemd
    [ 1468.282647] [   1434]     0  1434     4802      343      309       34         0    73728        0           100 (sd-pam)
    [ 1468.294330] [   1439]     0  1439     1418      546      512       34         0    45056        0             0 sh
    [ 1468.315212] [   1548]     0  1548    34386      525      480       45         0   102400        0             0 vx_app_arm_remo
    [ 1468.372315] [   1589]     0  1589     3842      192      192        0         0    69632        0             0 systemd-userwor
    [ 1468.424516] [   1590]     0  1590     3842      223      192       31         0    69632        0             0 systemd-userwor
    [ 1468.482451] [   1591]     0  1591     3842      217      192       25         0    69632        0             0 systemd-userwor
    [ 1468.509532] [   1617]     0  1617    34386      497      480       17         0   102400        0             0 vx_app_arm_remo
    [ 1468.594925] [   1620]     0  1620      916       32       32        0         0    40960        0             0 run_demo.sh
    [ 1468.643949] [   1621]     0  1621  1047287   597945   597945        0         0  6635520        0             0 python3
    [ 1468.655262] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=system-dropbear.slice,mems_allowed=0,global_oom,task_memcg=/system.slice/system-dropbear.slice/dropbear@1-192.168.1.228:22-192.168.1.147:56638.service,task=python3,pid=1621,uid=0
    [ 1468.695332] Out of memory: Killed process 1621 (python3) total-vm:4189148kB, anon-rss:2391780kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:6480kB oom_score_adj:0
    

    root@8374745b2c72:/home/root/examples/osrt_python/ort# python3 onnxrt_ep.py -c -m de-7300_onnxrt_fast-depth
    Skipping import of model optimizer
    Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
    
    Running 1 Models - ['de-7300_onnxrt_fast-depth']
    
    
    Running_Model :  de-7300_onnxrt_fast-depth  
    
    
    Running shape inference on model ../../../models/public/fast-depth.onnx 
    
    ========================= [Model Compilation Started] =========================
    
    Model compilation will perform the following stages:
    1. Parsing
    2. Graph Optimization
    3. Quantization & Calibration
    4. Memory Planning
    
    ============================== [Version Summary] ==============================
    
    -------------------------------------------------------------------------------
    |          TIDL Tools Version          |              11_00_06_00             |
    -------------------------------------------------------------------------------
    |         C7x Firmware Version         |              11_00_00_00             |
    -------------------------------------------------------------------------------
    |            Runtime Version           |                1.15.0                |
    -------------------------------------------------------------------------------
    |          Model Opset Version         |                  11                  |
    -------------------------------------------------------------------------------
    
    ============================== [Parsing Started] ==============================
    
    [TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options
    
    ------------------------- Subgraph Information Summary -------------------------
    -------------------------------------------------------------------------------
    |          Core           |      No. of Nodes       |   Number of Subgraphs   |
    -------------------------------------------------------------------------------
    | C7x                     |                      84 |                       1 |
    | CPU                     |                       0 |                       x |
    -------------------------------------------------------------------------------
    ============================= [Parsing Completed] =============================
    
    ==================== [Optimization for subgraph_0 Started] ====================
    
    ----------------------------- Optimization Summary -----------------------------
    --------------------------------------------------------------------------------
    |         Layer         | Nodes before optimization | Nodes after optimization |
    --------------------------------------------------------------------------------
    | TIDL_EltWiseLayer     |                         3 |                        3 |
    | TIDL_ReLULayer        |                        11 |                        0 |
    | TIDL_CLipLayer        |                        27 |                        0 |
    | TIDL_ResizeLayer      |                         5 |                        5 |
    | TIDL_ConvolutionLayer |                        38 |                       38 |
    --------------------------------------------------------------------------------
    
    Total nodes in subgraph: 50
    
    =================== [Optimization for subgraph_0 Completed] ===================
    
    The soft limit is 10240
    The hard limit is 10240
    MEM: Init ... !!!
    MEM: Init ... Done !!!
     0.0s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
     0.8s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
     0.12s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
     0.153s:  VX_ZONE_INFO: [ownAddTargetKernelInternal:189] registered kernel vx_tutorial_graph.phase_rgb on target DSP_C7-1
     0.181s:  VX_ZONE_INFO: [ownAddTargetKernelInternal:189] registered kernel vx_tutorial_graph.phase_rgb on target DSP_C7-2
     0.1148s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 
     0.1182s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 
     0.1222s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 
     0.1248s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 
     0.1277s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1 
     0.1307s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_2 
     0.1334s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_3 
     0.1360s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_4 
     0.1393s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_5 
     0.1419s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_6 
     0.1453s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_7 
     0.1483s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_8 
     0.1515s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2 
     0.1545s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_2 
     0.1575s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_3 
     0.1604s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_4 
     0.1629s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_5 
     0.1678s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_6 
     0.1713s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_7 
     0.1742s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_8 
     0.1781s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MCU2-0 
     0.1810s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_LDC1 
     0.1837s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_MSC1 
     0.1867s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_MSC2 
     0.1893s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_VISS1 
     0.1924s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE1 
     0.1947s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE2 
     0.1970s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE3 
     0.1995s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE4 
     0.2018s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DISPLAY1 
     0.2041s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DISPLAY2 
     0.2067s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CSITX 
     0.2091s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CSITX2 
     0.2114s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DMPAC_SDE 
     0.2138s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DMPAC_DOF 
     0.2164s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_FC 
     0.2194s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MCU1-0 
     0.2197s:  VX_ZONE_INFO: [tivxInit:152] Initialization Done !!!
     0.2202s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
    ============= [Quantization & Calibration for subgraph_0 Started] =============
    
    
    -------- Running Calibration in Float Mode to Collect Tensor Statistics --------
    [=============================================================================] 100 %
    
    ------------------ Fixed-point Calibration Iteration [1 / 5]: ------------------
    [=============================================================================] 100 %
    
    ------------------ Fixed-point Calibration Iteration [2 / 5]: ------------------
    [=============================================================================] 100 %
    
    ------------------ Fixed-point Calibration Iteration [3 / 5]: ------------------
    [=============================================================================] 100 %
    
    ------------------ Fixed-point Calibration Iteration [4 / 5]: ------------------
    [=============================================================================] 100 %
    
    ------------------ Fixed-point Calibration Iteration [5 / 5]: ------------------
    [=============================================================================] 100 %
    
    ==================== [Quantization & Calibration Completed] ====================
    
    ========================== [Memory Planning Started] ==========================
    
    
    ------------------------- Network Compiler Traces ------------------------------
    Successful Memory Allocation
    Successful Workload Creation
    
    ========================= [Memory Planning Completed] =========================
    
    Rerunning network compiler...
    ========================== [Memory Planning Started] ==========================
    
    
    ------------------------- Network Compiler Traces ------------------------------
    Successful Memory Allocation
    Successful Workload Creation
    
    ========================= [Memory Planning Completed] =========================
    
    ======================== Subgraph Compiled Successfully ========================
    
    
    
    
     
    Completed_Model :     1, Name : de-7300_onnxrt_fast-depth                         , Total time :    3339.03, Offload Time :     265.37 , DDR RW MBs : 0, Output Image File : py_out_de-7300_onnxrt_fast-depth_ADE_val_00001801.jpg, Output Bin File : py_out_de-7300_onnxrt_fast-depth_ADE_val_00001801.bin
     
     
    MEM: Deinit ... !!!
    MEM: Alloc's: 26 alloc's of 78380273 bytes 
    MEM: Free's : 26 free's  of 78380273 bytes 
    MEM: Open's : 0 allocs  of 0 bytes 
    MEM: Deinit ... Done !!!
    


    root@j722s-evm:/opt# ./edgeai-dl-inferer/bin/Release/dump_model_info -m /opt/model_zoo/de-7300_onnxrt_fast-depth 
    libtidl_onnxrt_EP loaded 0xaaab0551cf30 
    Final number of subgraphs created are : 1, - Offloaded Nodes - 84, Total Nodes - 84 
    APP: Init ... !!!
       728.421585 s: MEM: Init ... !!!
       728.421672 s: MEM: Initialized DMA HEAP (fd=5) !!!
       728.421905 s: MEM: Init ... Done !!!
       728.421935 s: IPC: Init ... !!!
       728.477950 s: IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
       728.491077 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
       728.502091 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
       728.502144 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
       728.502163 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
       728.503978 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 
       728.504253 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 
       728.504456 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 
       728.504629 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 
       728.504656 s:  VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!!
       728.504687 s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
       728.584085 s: IPC: Deinit ... !!!
       728.585943 s: IPC: DeInit ... Done !!!
       728.586026 s: MEM: Deinit ... !!!
       728.586052 s: DDR_SHARED_MEM: Alloc's: 7 alloc's of 4443780 bytes 
       728.586069 s: DDR_SHARED_MEM: Free's : 7 free's  of 4443780 bytes 
       728.586077 s: DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
       728.586100 s: MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    

  • Hi Andow,

    Did you re-compile your model on the host and replace the artifacts with your model?  If not the artifacts are from the old model and the model and configuration is for the new model.  I do not know anything about the repository your are looking into and it have has been updated for a couple of years.  I would suspect there is some bit-rot also.  You will have to contact the owner of the  https://github.com/TexasInstruments-Sandbox/edgeai-demo-monodepth-estimation repository for details.

    Regards,

    Chris 

  • Yes, I recompiled the model on the host and replaced the entire folder on the board.

    Is this the same project or are they different authors? It looks like they all use the same technology.
    https://dev.ti.com/tirex/explore/content/edge_ai_demos_01_04_06_04/demos/AI_based_Monocular_Depth_Estimation.html#how-to-get-started
    software-dl.ti.com/.../group_apps_cv_demos_app_sfm.html

  • Hi Andow,

    Have you tried running your model in TIDLRT or OSRT?  If it ran OK, I will add the author of demo to this thread.  

    Regards,

    Chris 

  • Thanks for the help, just checked with OSRT and found the error

        "de-7300_onnxrt_fast-depth": create_model_config(
               task_type="depth_estimation",
               source=dict(
                   model_url="http://software-dl.ti.com/jacinto7/esd/modelzoo/11_00_00/models/vision/depth_estimation/nyudepthv2/fast-depth/fast-depth.onnx",
                   infer_shape=True,
               ),
               preprocess=dict(
                   resize=[246, 246], 
                   crop=[224, 224],
                   data_layout="NCHW",
                   pad_color=0,
                   resize_with_pad=False,
                   reverse_channels=False,
                   add_flip_image=False, 
               ),
               session=dict(
                   session_name="onnxrt",
                   model_path=os.path.join(models_base_path, "fast-depth.onnx"),
                   target_device="AM67A", 
                   input_mean=[0.0, 0.0, 0.0],
                   input_scale=[0.003922, 0.003922, 0.003922], 
                   input_optimization=False, 
               ),
               postprocess=dict(
                   with_argmax=False, 
               ),
               extra_info=dict(
                   num_images=numImages, 
                   num_classes=1, 
               ),
           ),

    root@8374745b2c72:/home/root/examples/osrt_python/ort# python3 onnxrt_ep.py -m de-7300_onnxrt_fast-depth
    Skipping import of model optimizer
    Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
    
    Running 1 Models - ['de-7300_onnxrt_fast-depth']
    
    
    Running_Model :  de-7300_onnxrt_fast-depth  
    
    Process Process-1:
    Traceback (most recent call last):
      File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
        self.run()
      File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
        self._target(*self._args, **self._kwargs)
      File "/home/root/examples/osrt_python/ort/onnxrt_ep.py", line 315, in run_model
        input_image = test_images
    UnboundLocalError: local variable 'test_images' referenced before assignment
    

  • root@8374745b2c72:/home/root/examples/osrt_python/ort# python3 onnxrt_ep.py -m de-7310_midas-small_onnx
    Skipping import of model optimizer
    Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
    
    Running 1 Models - ['de-7310_midas-small_onnx']
    
    
    Running_Model :  de-7310_midas-small_onnx  
    
    libtidl_onnxrt_EP loaded 0x57dd0bfb4f10 
    Final number of subgraphs created are : 1, - Offloaded Nodes - 84, Total Nodes - 84 
    The soft limit is 10240
    The hard limit is 10240
    MEM: Init ... !!!
    MEM: Init ... Done !!!
     0.0s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
     0.16s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
     0.18s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
     0.136s:  VX_ZONE_INFO: [ownAddTargetKernelInternal:189] registered kernel vx_tutorial_graph.phase_rgb on target DSP_C7-1
     0.157s:  VX_ZONE_INFO: [ownAddTargetKernelInternal:189] registered kernel vx_tutorial_graph.phase_rgb on target DSP_C7-2
     0.865s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 
     0.902s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 
     0.927s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 
     0.956s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 
     0.983s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1 
     0.1019s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_2 
     0.1045s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_3 
     0.1068s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_4 
     0.1090s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_5 
     0.1118s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_6 
     0.1153s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_7 
     0.1172s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-1_PRI_8 
     0.1202s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2 
     0.1223s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_2 
     0.1243s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_3 
     0.1298s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_4 
     0.1318s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_5 
     0.1338s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_6 
     0.1364s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_7 
     0.1387s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DSP_C7-2_PRI_8 
     0.1414s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MCU2-0 
     0.1437s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_LDC1 
     0.1462s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_MSC1 
     0.1488s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_MSC2 
     0.1511s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_VISS1 
     0.1543s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE1 
     0.1563s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE2 
     0.1586s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE3 
     0.1611s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CAPTURE4 
     0.1631s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DISPLAY1 
     0.1653s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DISPLAY2 
     0.1679s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CSITX 
     0.1711s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target CSITX2 
     0.1751s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DMPAC_SDE 
     0.1788s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target DMPAC_DOF 
     0.1831s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target VPAC_FC 
     0.1862s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MCU1-0 
     0.1865s:  VX_ZONE_INFO: [tivxInit:152] Initialization Done !!!
     0.1870s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
    
    Saving image to  ../../../output_images/
    
    Saving output tensor to  ../../../output_binaries/
    
     
    Completed_Model :     1, Name : de-7310_midas-small_onnx                          , Total time :     253.18, Offload Time :     253.11 , DDR RW MBs : 0, Output Image File : py_out_de-7310_midas-small_onnx_pexels-olly-762020.jpg, Output Bin File : py_out_de-7310_midas-small_onnx_pexels-olly-762020.bin
     
     
    MEM: Deinit ... !!!
    MEM: Alloc's: 26 alloc's of 25004435 bytes 
    MEM: Free's : 26 free's  of 25004435 bytes 
    MEM: Open's : 0 allocs  of 0 bytes 
    MEM: Deinit ... Done !!!
    


    okay, fixed. 
    Are there any examples of integrating depth estimation for model testing?
    My vision application does not have Structure From Motion Application...

  • Hi Andow,

    Please ask this question in a new thread.  These threads are searched as part of our knowledge base and we like to keep them to one topic.

    Regards,

    Chris