TDA4VM: TIDL delegate with multiple subgraphs

Sharon Ngu

Part Number: TDA4VM

Hi,

According to https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_02_00_05/exports/docs/tidl_j721e_08_02_00_11/ti_dl/docs/user_guide_html/md_tidl_fsg_quantization.html , we should be using floating point model as input. If there are multiple subgraphs, I believe the ones offloaded to C7x would be in fixed point. What about the parts of the model which are running on ARM - would they be in floating point or fixed point? If this is in floating point, would you advise to use a quantized tflite model as input?

Thanks.

Kind regards,

Sharon

over 3 years ago

0 kumar.desappan over 3 years ago

TI__Mastermind 22145 points

if the original model is in floating-point version, then the portion of the model running in ARM be computed with Floating-point mode.

Yes, the quantized Tflite model would give better latency compared to the floating-point model.

0 Sharon Ngu over 3 years ago in reply to kumar.desappan

Prodigy 30 points

Thanks for the response. If we use a quantized TFlite model as input, would that imply we are doing quantization twice? From floating point TFLite -> quantized TFLite -> quantized TI. Would you have any measurement on how this affect performance in terms of accuracy?