Support for BatchNorm1d

Sourabh Patil

Other Parts Discussed in Thread: TDA4VM

Hi team,

I was going through the supported operations and found that there is support for "BatchNormalization". I have my custom model in which there is BatchNorm layer which applies 1d batch norm (to an output from Linear layer). When I compile model and see runtimes_visualization.svg, I find that BatchNorm is TIDL supported (also could confirm it from debug_level 4). But, output is not as expected (dimension wise, expected is 1x512 and I get a single value so dim is empty tuple). So, I guess it is supported only for 2d BatchNorm and not 1d Batchnorm even though it virtually goes on the board for acceleration. Will there be a future update regarding this? I had one more workaround in my mind, as currently supported BN works channel wise, should I change my 1d vector (1x512) to 4d 1x512x1x1 and try to compile it?

Thanks,
Sourabh

over 1 year ago

0 LeonardEllis over 1 year ago

TI__Guru 71526 points

Hello Sourabh,

Please provide more information - what processor are you using?

Thank you,

~Leonard

0 Sourabh Patil over 1 year ago in reply to LeonardEllis

Prodigy 146 points

Hello, I am using TI TDA4VM J721E board, with SDK version 09_00_00_00. I have one more follow up question, the output from onnxruntime session on my system should match with output taken on TDA4VM board? The dynamic range of values is different, in my case.

0 Pratik Kedar over 1 year ago in reply to Sourabh Patil

TI__Mastermind 24041 points

Hi,

You can find the list of supported operators here : https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/supported_ops_rts_versions.md

From the network compiler side, batch norm usually get absorb in subsequent layers like convolution etc.

Please let me know this if list doesn't consist of the operator you are looking for then that operator is not supported as part of TIDL package.
Also if the operator you are looking for doesn't found in above list, can you explain the required kernels requirement wrt current implementation of the kernel in details that would helpful to drive the conversation ahead.

0 Sourabh Patil over 1 year ago in reply to Pratik Kedar

Prodigy 146 points

The issue is there is difference between Batchnorm2d and Batchnorm1d, and the way it accepts the inputs. Batchnorm2d expects 4 D shape (which is supported according to the documentation) but if custom model uses Batchnorm1d at some point then it fails to give expected output after compilation (even though it shows that its is supported, from debug message as well as runtime.svg). But there is work around which I think might work, to expand the dimensions and align the features with channel dimension.

0 Pratik Kedar over 1 year ago in reply to Sourabh Patil

TI__Mastermind 24041 points

Let me loop in out TILD expert for the same.

0 Varun Tripathi over 1 year ago in reply to Sourabh Patil

TI__Genius 10055 points

Sourabh,
Your interpretation is correct - we will update our documentation to reflect this (We have support for 4D (NxCxHxW) Batchnorm and not generic ND (NxCxD1xD2).. Batchnorm). Our recommendation would be the same as your workaround (expand dimensions to have the channel dimension outside)

Thanks,
Varun

Arm-based microcontrollers

Arm-based microcontrollers forum

Support for BatchNorm1d