J784S4XEVM: 32 bit vs. 16 bit vs. 8 bit

Anna Tao

Part Number: J784S4XEVM
Other Parts Discussed in Thread: TDA4VH, AM69A

Hi, we are looking to run our model on the tda4vh chip. Currently our model is quantized for 32 bits. I have read on the edge-ai tools repo that potentially the chip and board will not support 32 bits. I was wondering if we put our 32 bit model onto the evm board, will it run and what types of truncation we can expect if it runs.

And for future planning, how does the performance speed and precision/accuracy of the 16 bit and 8 bit models compare. For example is it worth quantizing down to 8 bit for that extra speed with less precision.

Some information about this would be greatly appreciated!

8 months ago

0 Christina Kuruvilla 7 months ago

TI__Expert 7120 points

Hi Anna,

Float32 is not supported to run on the TDA4VH with TIDL on board. However, you can run a model that is originally quantized to 32 bits, through TIDL's PC model compilation, to create the model artifacts that will run on 8 bit or 16 bit on board.

When comparing 16 bit versus 8bit, depending on the size of your model and data inputs, you will see slight precision drop with the 8bit, with higher speed, and higher precision with the 16 bit with slower times.

Below are some helpful links from the edgeai-tidl-tools on how to debug any accuracy issues + quantization

https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_osr_debug.md

https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_fsg_quantization.md

I do recommend quantizing to 8 bit on ONNXruntime, without any involvement with TIDL, to help you optimize if you see that the performance between 16 bit and 8 bit are significant. We recommend this, so you can see which areas are due to the expected comparison between 16 bit and 8 bit, and which are significant enough to be considered a bug in TIDL.

If you have any further questions, please let me know.

Warm regards,

Christina Kuruvilla

0 Anna Tao 7 months ago in reply to Christina Kuruvilla

Prodigy 50 points

Awesome, a followup question I have was how "tensor_bits" worked in the provider options when I want to run on the evm board.

If I set this to 16 or 8 would it behave any differently?

provider_options = {
"tidl_tools_path": os.environ["TIDL_TOOLS_PATH"],
"artifacts_folder": "tidl_artifacts",
"tensor_bits": 32,
"advanced_options:inference_mode": 0,
"advanced_options:num_cores": 4,
}

sess = rt.InferenceSession(
model_path,
providers=[ "TIDLExecutionProvider","CPUExecutionProvider"],
provider_options=[provider_options, {}],
)

0 Christina Kuruvilla 7 months ago in reply to Anna Tao

TI__Expert 7120 points

Hi Anna,

Yes so the tensor bits is how to set which bit depth the model will run in during inference in TIDL. Currently, you have 32 bit, which will only run on PC emulation inference. 16 bit and 8 bit will both run on the device inference (and on PC emulation inference), but have the slight expected differences as mentioned earlier.

I hope this made sense, let me know if you have any further questions.

Warm regards,

Christina

0 Anna Tao 7 months ago in reply to Christina Kuruvilla

Prodigy 50 points

Hi Christina,

Thank you so much for the clarification!

I am getting these kernel initialization errors. Do you know why they are happening and what I can do to fix them?

0 Christina Kuruvilla 7 months ago in reply to Anna Tao

TI__Expert 7120 points

Hi Anna,

Apologies for the delay in getting back to you. Which version of TIDL did you use to compile the artifacts? And what version of the SDK are you using?

This seems to be a mismatch between the versions. You can check the compatible versions here: https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/sdk_version_compatibility_table.md

Warm regards,
Christina

0 Anna Tao 7 months ago in reply to Christina Kuruvilla

Prodigy 50 points

Hi Christina,

Thank you for responding!

To compile we created our own script that fed in our scenario data and ran sess.run to calibrate. we exported SOC=am69a, ran the setup script, entered the docker environment, and compiled under the edge-ai-tools repository to produce those artifacts.

Currently I am not running with an SDK? I'm not super sure if this is correct, but i put the edge-ai-tools repository onto the board's SD card along with the pre-compiled artifacts, then tried to run the following script on the board:

provider_options = {
"tidl_tools_path": os.environ["TIDL_TOOLS_PATH"],
"artifacts_folder": "tidl_artifacts",
"tensor_bits": 32,
"advanced_options:inference_mode": 0,
"advanced_options:num_cores": 4,
}

sess = rt.InferenceSession(
model_path,
providers=[ "TIDLExecutionProvider","CPUExecutionProvider"],
provider_options=[provider_options, {}],
)
print("Providers provided:", sess.get_providers())

# inference test
inputs = {
"w": np.zeros((1, 1, 1, 1, 1, 4), dtype=np.float32),
"x": np.random.randn(1, 1, 1, 1, 1, 64).astype(np.float32),
"y": np.full((1, 1, 1, 1, 1), 2.5, dtype=np.float32),
"z": np.zeros((1, 1, 1, 1, 2, 64), dtype=np.float32),
}


outputs = sess.run(None, inputs)

0 Christina Kuruvilla 7 months ago in reply to Anna Tao

TI__Expert 7120 points

Hi Anna,

It is a bit confusing especially if backward compatible versions come into play. Which version of the board's SD card did you flash? This is the SDK version. For the edgeai-tidl-tools version, this would be whichever tag you checked out too. Alternatively, you can also see which version of edgeai-tidl-tools is being run during the log output in compile. The version of edgeai-tidl-tools, and the board's image version (SDK version) needs to be compatible to run on the board properly.

These two information are also vital for me to recreate the behavior you are seeing as well.

Warm regards,

Christina

0 Anna Tao 7 months ago in reply to Christina Kuruvilla

Prodigy 50 points

Thank you, I understand your point

So, I personally did not setup the board's SD card. My coworker who was in charge of this told me the SD card seemed to already be set-up when it arrived in the mail...

Is there any way we can check which version of the board's SD card was flashed.

Your responses are greatly appreciated.

0 Christina Kuruvilla 7 months ago in reply to Anna Tao

TI__Expert 7120 points

Hi Anna,

Could you run this command

ls /usr/lib/ | grep -i libtivision

It will show which version of the SDK you are running.

Warm regards,

Christina

0 Anna Tao 7 months ago in reply to Christina Kuruvilla

Prodigy 50 points

Hi Christina,

Thank you for that

So I don't see any SDK related output from that command this is all i see:

root@j784s4-evm:~# ls /usr/lib/ | grep -i libtivision
libtivision_apps.so
libtivision_apps.so.10.0.0
root@j784s4-evm:~#

is the SDK version 10.0.0?

0 Anna Tao 7 months ago in reply to Anna Tao

Prodigy 50 points

So we used the latest/default branch on the edge-ai-tools repo, which perhaps is a version of 11.x.x. Is it backwards compatible?

0 Christina Kuruvilla 7 months ago in reply to Anna Tao

TI__Expert 7120 points

Hi Anna,

Anna Tao said:
is the SDK version 10.0.0?

Yes this shows that you are using the 10.0 SDK version, which is not compatible with the latest version of the edgeai-tidl-tools repo (11.2 currently).

There are two things that can be done here. You can upgrade your SDK version to the latest one, though this would take some more time as it requires building the SDK (less if you are able to use the prebuilt SDK), however this will help you in the long run as there have been many bug fixes and optimizations since the 10.0 SDK. I would recommend doing this method so you don't waste time debugging a known and fixed issue.

The link to the SDK can be found here: https://www.ti.com/tool/download/PROCESSOR-SDK-RTOS-J784S4

The other is to use the link shared earlier with the corresponding edgeai-tidl-tools version to create the compatible artifacts. This is the fastest method out of the two however, could result in the bugs mentioned earlier. You can use any of these versions, but I recommend using the backwards compatible of 10.1 that is shown at the top of the list to get past at least some of the bugs.

If you need any more instructions on how to do either, please let me know.

Warm regards,

Christina

0 Anna Tao 7 months ago in reply to Christina Kuruvilla

Prodigy 50 points

Hi Christina,

As an update:
We git checked out to the correct version and recompiled the artifacts with the correct edge-ai-tools repo branch. Yet still i am getting similar errors.

Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
libtidl_onnxrt_EP loaded 0x51c8640 
Final number of subgraphs created are : 9, - Offloaded Nodes - 96, Total Nodes - 104 
APP: Init ... !!!
    89.768620 s: MEM: Init ... !!!
    89.768666 s: MEM: Initialized DMA HEAP (fd=5) !!!
    89.768790 s: MEM: Init ... Done !!!
    89.768811 s: IPC: Init ... !!!
    89.794934 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
    89.801644 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
    89.801743 s:  VX_ZONE_INIT:Enabled
    89.801754 s:  VX_ZONE_ERROR:Enabled
    89.801766 s:  VX_ZONE_WARNING:Enabled
    89.802408 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0 
    89.802560 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1 
    89.802666 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2 
    89.802775 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3 
    89.802786 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
    89.803418 s:  VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
    89.810008 s:  VX_ZONE_ERROR:[ownContextSendCmd:885] Command ack message returned failure cmd_status: -1
    89.810041 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    89.810051 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
    89.810060 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    89.810076 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:2:2 ... failed !!!
    89.810099 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
    89.810109 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
    89.817726 s:  VX_ZONE_ERROR:[ownContextSendCmd:885] Command ack message returned failure cmd_status: -1
    89.817759 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    89.817769 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
    89.817778 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    89.817791 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:1:1 ... failed !!!
    89.817802 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
    89.817810 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
    89.825506 s:  VX_ZONE_ERROR:[ownContextSendCmd:885] Command ack message returned failure cmd_status: -1
    89.825541 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    89.825552 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
    89.825560 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    89.825572 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:2:2 ... failed !!!
    89.825584 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
    89.825595 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
    89.831816 s:  VX_ZONE_ERROR:[ownContextSendCmd:885] Command ack message returned failure cmd_status: -1
    89.831847 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    89.831854 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
    89.831871 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    89.831883 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:1:2 ... failed !!!
    89.831896 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
    89.831906 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
    89.838402 s:  VX_ZONE_ERROR:[ownContextSendCmd:885] Command ack message returned failure cmd_status: -1
    89.838432 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    89.838442 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
    89.838453 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    89.838465 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:2:1 ... failed !!!
    89.838479 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
    89.838488 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
    89.846119 s:  VX_ZONE_ERROR:[ownContextSendCmd:885] Command ack message returned failure cmd_status: -1
    89.846151 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    89.846158 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
    89.846168 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    89.846180 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:4:3 ... failed !!!
    89.846192 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
    89.846201 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
    89.852045 s:  VX_ZONE_ERROR:[ownContextSendCmd:885] Command ack message returned failure cmd_status: -1
    89.852078 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    89.852085 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
    89.852095 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    89.852108 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:2:1 ... failed !!!
    89.852119 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
    89.852127 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
    89.858318 s:  VX_ZONE_ERROR:[ownContextSendCmd:885] Command ack message returned failure cmd_status: -1
    89.858349 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    89.858356 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
    89.858366 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    89.858377 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:2:1 ... failed !!!
    89.858393 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
    89.858404 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
    89.866516 s:  VX_ZONE_ERROR:[ownContextSendCmd:885] Command ack message returned failure cmd_status: -1
    89.866548 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    89.866559 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
    89.866567 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    89.866580 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:8:2 ... failed !!!
    89.866592 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
    89.866601 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
    97.458475 s:  VX_ZONE_ERROR:[ownContextSendCmd:885] Command ack message returned failure cmd_status: -1
    97.458514 s:  VX_ZONE_ERROR:[ownNodeKernelInit:592] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    97.458526 s:  VX_ZONE_ERROR:[ownNodeKernelInit:593] Please be sure the target callbacks have been registered for this core
    97.458537 s:  VX_ZONE_ERROR:[ownNodeKernelInit:594] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    97.458549 s:  VX_ZONE_ERROR:[ownGraphNodeKernelInit:620] kernel init for node 0, kernel com.ti.tidl:2:2 ... failed !!!
    97.458569 s:  VX_ZONE_ERROR:[vxVerifyGraph:2254] Node kernel init failed
    97.458579 s:  VX_ZONE_ERROR:[vxVerifyGraph:2311] Graph verify failed
    97.458647 s:  VX_ZONE_ERROR:[ownGraphScheduleGraphWrapper:919] graph is not in a state required to be scheduled
    97.458656 s:  VX_ZONE_ERROR:[vxProcessGraph:844] schedule graph failed
    97.458664 s:  VX_ZONE_ERROR:[vxProcessGraph:849] wait graph failed
ERROR: Running TIDL graph ... Failed !!!
2026-01-29 19:35:20.044781288 [E:onnxruntime:, sequential_executor.cc:494 ExecuteKernel] Non-zero status code returned while running TIDL_0 node. Name:'TIDLExecutionProvider_TIDL_0_0' Status Message: TIDL Compute Invoke Failed.
Traceback (most recent call last):
  File "/root/tidl_itra_integration_10_01_03_00/evm_my_script.py", line 165, in <module>
    model_outputs = action_model.forward(
                    ^^^^^^^^^^^^^^^^^^^^^
  File "/root/tidl_itra_integration_10_01_03_00/onnx_wrapper.py", line 44, in forward
    outputs = self.session.run(None, inputs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 200, in run
    return self._sess.run(output_names, input_feed, run_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running TIDL_0 node. Name:'TIDLExecutionProvider_TIDL_0_0' Status Message: TIDL Compute Invoke Failed.
    97.700674 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:120] De-Initialization Done for HOST !!!
    97.705282 s:  VX_ZONE_INIT:[tivxDeInitLocal:206] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
    97.706170 s: IPC: Deinit ... !!!
    97.706858 s: IPC: DeInit ... Done !!!
    97.706876 s: MEM: Deinit ... !!!
    97.706887 s: DDR_SHARED_MEM: Alloc's: 85 alloc's of 18816838 bytes 
    97.706895 s: DDR_SHARED_MEM: Free's : 85 free's  of 18816838 bytes 
    97.706901 s: DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
    97.706912 s: MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!

I am not sure what is happening and how I can diagnose this issue. Is it possible the setup of the SD card was done incorrectly?

0 Christina Kuruvilla 7 months ago in reply to Anna Tao

TI__Expert 7120 points

Hi Anna,

Yes since you mentioned that this SD card was setup before it got to you, it is possible it was not setup with the necessary TIDL dependencies. I would recommend building and flashing a new SD card. If you have any questions regarding this process, please let me know.

Also, before you even get to running on the device, I noticed in your log that your model was split into 9 subgraphs, meaning there are about 8 unsupported layers in your model that run outside of TIDL. I recommend optimizing this either in parallel or before building the new SD card. During compilation, the log should tell you which layers are not compatible with TIDL.

Warm regards,

Christina Kuruvilla

0 Anna Tao 7 months ago in reply to Christina Kuruvilla

Prodigy 50 points

Hi Christina,

I have flashed version 11 of the SDK onto my SD card (I forget which version specifically but I made sure to match the compatible SDK version with my current tidl-edgeai-tools version). Now all the operations of my model are supported, but when I try to run it I get this output:

Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']
libtidl_onnxrt_EP loaded 0x30a3ba60 
Final number of subgraphs created are : 1, - Offloaded Nodes - 104, Total Nodes - 104 
APP: Init ... !!!
   490.439328 s: MEM: Init ... !!!
   490.439374 s: MEM: Initialized DMA HEAP (fd=5) !!!
   490.439513 s: MEM: Init ... Done !!!
   490.439535 s: IPC: Init ... !!!
   490.466859 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   490.474285 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   490.474380 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
   490.474394 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
   490.474405 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
   490.475069 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 
   490.475191 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 
   490.475296 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 
   490.475417 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 
   490.475435 s:  VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!!
   490.475456 s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
Illegal instruction (core dumped)

I suspect that it's due to running x86 binary on ARM architecture, but am not sure where this is from or how to fix this. Do you have any advice?

P.S: Here is the output when I run "file tools/AM69A/tidl_tools/libtidl_onnxrt_EP.so":

tools/AM69A/tidl_tools/libtidl_onnxrt_EP.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=147d61db071bb88e93dfbef57b1edcea4c065a27, not stripped

0 Christina Kuruvilla 7 months ago in reply to Anna Tao

TI__Expert 7120 points

Hi Anna,

Is the modified model completing inference on PC for you? I want to double check if your SDK was built with all the necessary dependencies, so could you run one of the models from the examples/model zoo to double check if it all got built?

I am also adding instructions on how to properly build just for you to double check below. There is also an example on how to run the demo to check the integrity of the build:

=================================

=== How to build j784s4 RTOS PSDK ===

=================================

1. In the SDK root directory, install untarred RTOS PSDK package

./sdk_builder/scripts/setup_psdk_rtos.sh --install_tidl_deps

2. setup the enviroment (or you can edit the file "sdk_builder/build_flags.mak" and modify below 4 variables)

export BUILD_EMULATION_MODE=no

export BUILD_TARGET_MODE=yes

export SOC=j784s4

export PSDK_INSTALL_PATH=$(pwd)

3. After building has been finished, make sure the $SOC has been set correctly

echo $SOC
it should show "j784s4" (or SOC selected)

4. Do below to build the full PSDK RTOS, with "N" being the number of parallel threads

cd sdk_builder
make sdk_scrub

make sdk -j16
this is the same as above one, but it will save an error message to a log file if any error occurs during the compilation
./make_sdk.sh

5. To do a clean build of vision_apps do below,

cd sdk_builder
make vision_apps_scrub

6. Do below to build vision-apps or tidl-pc-tools (which will generate tidl_importer.out etc.)

make vision_apps
make tidl_rt

make tidl_pc_tools

============================================================

== Build a boot-able SD card based on RTOS for J7xxx device ==

============================================================

Assuming you have completed the building of a RTOS-SDK using: make sdk

1. Use the command "df -h" to identify the device used by SD card

df -h

2. Unmount the SD card before running the script

Example, if your SD card is on device /dev/TEST having two partitions, do below to unmount them (change TEST to correct correlating)
umount /dev/TESTBOOT

umount /dev/TESTrootfs

3. Run the script as below to partition it, answer "y" to all questions asked when running the script in ${PSDKR_PATH}

sudo sdk_builder/scripts/mk-linux-card.sh /dev/TEST

4. Eject and re-insert the SD card.

df -h

5. Copy/untar filesystem and boot files to SD card by executing the script as shown below in ${PSDKR_PATH}

sdk_builder/scripts/install_to_sd_card.sh

6. Add the ti data set

cd /media/$USER/rootfs/
mkdir -p opt/vision_apps
cd opt/vision_apps
 
tar --strip-components=1 -xf ${path/to/file}/psdk_rtos_ti_data_set_xx_xx_xx.tar.gz
tar --strip-components=1 -xf ${path/to/file}/psdk_rtos_ti_data_set_xx_xx_xx_{SOC}.tar.gz
sync

7. Go back to your SDK directory

cd ${PSDKR_PATH}/sdk_builder

make linux_fs_install_sd
This also copies a file uEnv.txt to boot partition to select the .dtbo's which is required to run vision apps demos.

8. Eject and remove SD card from PC and insert it into the EVM

9. On the EVM, do the below steps to init the environment for the demos

cd /opt/vision_apps

source ./vision_apps_init.sh

Run a TIDL example to have a display on the screen.

./run_app_tidl.sh

./run_app_tidl_od.sh

(this step will generate the "run_app_tidl.sh" etc. if you have copy the tidl test data)

Warm regards,

Christina Kuruvilla

0 Anna Tao 7 months ago in reply to Christina Kuruvilla

Prodigy 50 points

Hello, thank you for your response!

The issue seemed to be that Pytorch was build for x86. Once we removed all pytorch it worked :)

Thanks again for all your help!!

0 Christina Kuruvilla 7 months ago in reply to Anna Tao

TI__Expert 7120 points

Perfect! Glad you were able to figure this out and you were able to resolve. I will go ahead and close this ticket. Feel free to open a new E2e if you have a new question.

Warm regards,

Christina

Processors

Processors forum

J784S4XEVM: 32 bit vs. 16 bit vs. 8 bit