TDA4AL-Q1: How to Port a Deep Learning Model with Multiple Inputs (Multiple Inputs : float vector and uint8 image)

Hi Wen Li,

Thank you for your response. However, I feel that my original questions were not fully addressed, and I would like to clarify further:

1. Support for Multiple Inputs:
   My deep learning model requires four inputs:

   • Two inputs are images (uint8).
   • Two inputs are float vectors (float32).

   During the TIDL_RT PTQ process, I noticed that only a single input (an image) seems to be supported.
   Could you clarify:

   • Is there a way to enable multiple inputs in TIDL_RT?
   • If multiple inputs are not directly supported, how can I merge or preprocess these inputs to make them compatible with the TI platform while maintaining model accuracy?

2. Data Format Conversion:
   You mentioned in your response:

   "The input format is the image data format. For the vector data, it will depend on your model data format. You may have to convert the interface data format."

   Could you provide more details about this? Specifically:

   • What data format is expected for non-image vector inputs?
   • If the float vector inputs (32-bit) need to be converted to uint8, how should this be done without losing critical precision in the model?
   • Does TIDL automatically handle this conversion, or do I need to preprocess the data before inference?

3. Examples or Documentation:
   Are there any examples or documentation that show how to handle models with mixed input types (images and vectors)? Such examples would be extremely helpful in resolving this issue.

I appreciate your guidance on these specific points. If additional details are needed, please let me know.

Thanks and regards,
MinWoo

Hello,

I have reorganized my questions for clarity:

1. Support for Multiple Inputs:

   • Can TIDL handle deep learning models with multiple inputs (e.g., two images)?

2. Input Format Adjustment:

   • My original inputs are in float32, but I plan to scale and convert these to uint16 for compatibility. Does TIDL support this approach, and are there any specific requirements or limitations for such input conversion?

3. PTQ Process for Multi-Input Models:

   • If both of the above are supported, could you please provide the specific PTQ process or steps for each case?

I appreciate your detailed response on these points.

Thank you,
MinWoo

Hello Minwoo Yoo,

Assuming you are using TIDL-RT interface

Support for Multiple Inputs

TIDL supports multiple inputs, you have to concatenate all the inputs together and provide a single file as input during model compilation

Input Format Adjustment:

You can adjust the following parameters to adjust the data formats

rawDataInElementType - this option should be appropriately set based on the data type your input has in the input file (Ex: 6 if float)

inElementType - This is the data type of the input that your model is expecting. This can be different from the type your input has in the input file. So, by setting both of these options correctly you can convert the input

Ex: setting rawDataInElementType = 6 and inElementType = 0

The input is present as float in the input file and we convert the input to uint8 (TIDL test bench has the data convert code to do this conversion)

You can also add a dataconvert layer in the network using options (addDataConvertToNet) - this way data conversion happens in DSP, otherwise it happens on ARM core.

PTQ Process for Multi-Input Models

No specific options needed to handle multi inputs, options mentioned above should suffice.

Documentation of TIDL-RT options is here: https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_05_00_11/exports/docs/tidl_j721e_08_05_00_16/ti_dl/docs/user_guide_html/md_tidl_model_import.html 

We advise you to use our OSRT interface to compile models - as this removes all of the manual option setting, the data conversion options are set by default when using OSRT. Following is the link for our edgeai-tidl-tools repo 

https://github.com/TexasInstruments/edgeai-tidl-tools/tree/master 

Let me know if you have any other questions.

Thanks,

Sai

Hello Minwoo;

Did the TI expert (Saideepak B.)'s answer your questions? If you still have questions, please let us know or submit a new ticket.

Thanks and regards

Wen Li