TDA4VM: Use of PerfSim tool during ONNX import

Hi Ross,

     Can you explain more about what modification you have done in TIDL?

Regards,

Anshu

Re-post as I mis-clicked resolve when I meant to click reply.

---------

Hi Anshu,

I have added support for a new layer type which we need in our application (and changed the internal version number as that's a version breaking change).

However, PerfSim then fails as it doesn't know about the new layer type.  I believe it's the only component left I need to update, so I am keen to get a copy of the source asap as this is the last thing preventing me testing my application on the target.

Thanks,

Ross

Anshu Jain Any update please?  The lack of ability to recompile this tool is currently blocking our development.

Many Thanks,

Ross

Hi Ross,

     Can you help us understand what kind of layers which are not supported in TIDL which you want to add?

Regards,
Anshu

Hi Anshu,

For now I have added Instance Normalisation.

However, I'm not ruling out making further breaking changes in the future.  The important point is not what I did this time, but what the process is for rebuilding TIDL in its entirety.

At present I don't have enough in the SDKs to rebuilt everything used in the TIDL import and inference processes, and that's what I need to be resolved.

Thanks,

Ross

Hi Ross, 
     For adding new layers to TIDL we have plugin function based mechanism which can be used. Custom layer documentation is part of addon package for Processor SDK RTOS and details to get the same can be found here :
https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_06_00_12/exports/docs/psdk_rtos/docs/user_guide/datasheet_sdk_j721e.html

Once this is installed the custom layer documentation can be found at following location:
tidl_xx_xx_xx_xx/ti_dl/docs/user_guide_nda_html/md_tidl_custom_layer.html

Regards,

Anshu         

I already made a request for that, but have received no reply

However, from what I could read, that has limitations on the sort of layers / shapes that can be added, hence - as stated above - I want to be able to rebuild TIDL and the import tools in their entirety, so that I have more flexibility to do what I may need to in the future.

Is there any reason PerfSim source is not provided?

Hi Ross,

     Custom layer have limitation only if you want to use data flow API's which abstracts the DMA programming from the user, otherwise it doesn;t have any restrictions. For custom layers without dataflow abstraction API's only uses input and output pointer allocated by PerfSim tool which are allocated based on input and output tensor volumes. Other than this user has control on how they want to do processing of any custom layer.

Regards,
Anshu

Ross,

Sorry for the delay. Your request for access has been granted.

Regards,
Stanley

Hi Anshu,

I've had a look at this, but as far as I can tell, the custom layer handlers are not called during ONNX import via the TIDLCompilationProvider (tidl_model_import_onnx.so).

Given the stand-alone C based importer (tidl_model_import.out) - which does call them - doesn't seem to produce output in the right format for use with the ONNX execution provider TIDLExecutionProvider (see the first paragraph on the original post in this thread), it looks like I'm still stuck.

It's not clear to me why there are two different import processes and how to get them to interoperate.

Cheers,

Ross

Hi Ross,

      Can you confirm if you are using TIDL-RT or Open source runtime interface? Currently custom layer is only supported via TIDL-RT interface.

Regards,

Anshu

Hi Anshu,

My modelling team is using the ONNX OSRT at the moment, and they want to preserve this as an interchange format between PyTorch / Matlab / Simulink and TIDL-RT.

This was also what I was getting at in my first paragraph in the first post in this thread - I was asking about the difference between the two import processes - but it hasn't really been explained.

If the custom layer framework isn't supported using ONNX API, then it goes back to needing PerfSim source code to add in new layer support, as originally requested.

Ross

Hi Ross,

      Can you tell how many layers which you are looking for are not available as part of TIDL?

Regards,

Anshu

Anshu Jain

Please can you answer the question I keep asking.  Where can I get the source code for the graph compiler PerfSim?  It looks like it should be in the SDK, but it is not.

I have looked further at the custom layer hooks, and although I can manage to get them called via the ONNX import by modifying the import library, they still don't seem suitable (I will start a separate thread for questions on those hooks) due to inability to pass biases through (only weights) or access scratch memory (TIDL_scratchBufAlloc), and it's unclear what happens to the weights during the graph compiler / quantisation (since the max pool example doesn't even use weights and biases, and passes a custom structure through in the weights pointer instead - which clearly cannot be modified by quantisation / compilation).

Regards,

Ross

Hi Ross,

Ross O'Connor said:

Where can I get the source code for the graph compiler PerfSim?  It looks like it should be in the SDK, but it is not.

   We do not release source of graph compiler , its only released in binary form.

Regards,

Anshu

Ok, thanks Anshu.

I will then advise against the use of TIDL-RT as a platform for our products for now, as without the layers required and without the capability to add them, we will not be able to proceed.

Cheers,

Ross

Hi Ross,

    For the purpose you are looking for you don't need graph compiler source code. Let us know what exactly is missing from the available source code which is blocking your progress.

Regards,

Anshu