TDA4VM: TIDL Batch Inference Issue

Hi Stefen,

>> we saw that all TI's batch inference examples have batch size equal to 2^n, so we would like to know is this some limitation?

We had tested batch processing only with batch size equal to 2^n, but any other batch size number also should work as such there is no limitation on the batch processing. 

Is it possible for you to share the model so that I can check at our end ?

Thanks,

Praveen

Hello,

I will check with our customer can we share the model and I will get back to you.

One more question - Our model has following format for input tensor: 5x3x192x192. So should batch number be embedded in the model or our model should have following format 1x3x192x192 and batch number set as 5 only during import process (in import configuration file, numBatches parameter)?

Best Regards,

Stefan.

Hi Stefen,

Both the configuration should work, however we only tested with second configuration, that is setting batch number >1 during import process.

Thanks,

Praveen 

Hello,

Can you write down your e-mail so I can share the model with you?

Best Regards,

Stefan.

Hi Stefan,

I will not be able to provide my e-mail in the public forum, please contact local FAE.

Thanks,

Praveen

Hi Stefan,

I could download the models, could you please share the import and infer config files used and also sample input used for batch processing.

Thanks,

Praveen

Hi Praveen,

I sent required files on e-mail.

Best Regards,

Stefan.

Hi Stefen,

I received the config files, I will check and come back with an update.

Thanks,

Praveen

Hi Stefen,

Below is the update on the 2 models shared,

1. model_1.onnx (batch model) : When we try to import this model and import was unsuccessful because it is having few layers for which we don't have batch processing support currently and hence it is stuck in the test bench on target. This is a limitation of batch processing feature.

2. model_2.onnx (is not batch model) : I could replicate the issue that outputs from PC simulation and target don’t match completely, I am debugging this issue and will come back with an update soon.

Thanks,

Praveen

Hi Stefen,

I am closing this thread here as we are following it over the email, please feel free to open this if you need further clarifications.

Thanks,

Praveen