TDA2E: Different SSD models & some TDA4 questions

WEI CHIH CHERN

Part Number: TDA2E
Other Parts Discussed in Thread: TDA2

Hello,

I've several questions regarding caffe ssd implementations:

1. I understand the resize_width & resize_height, but what does the ssd_size referring to? Does it refer to the feature map size output by the base network (right after feature extraction)?

2. I've a relatively simpler detection task than ADAS, and I discovered that aside from ssdJacintoNetV2, in the code we also have jdetnet21_fpn, jdetnet21_s8, jdetnet21. Speed is also important to my application, does the remaining 3 networks offer any speed boost over ssdJacintoNetV2?

3. My boss has been considering to use TDA4, and I see one of the TI engineer mentioned the caffe trained networks are expected to work on TDA4. Does the conversion tool and usecase are provided already for TDA4 to run caffe models with hardware acceleration? If not, when would it be ready?

4. A TI engineer also mentioned, L1 & sparse training offer no speed boost on TDA4, would you please kindly to share some benchmark stats? How come including zeros to models to speed up calculation doesn't end up making much difference on TDA4? Would you please kindly explain the reason?

Thank you,

Wei Chih

over 5 years ago

0 kumar.desappan over 5 years ago

TI__Mastermind 22145 points

1. Are you referring to import config "resize_width & resize_height" if yes, this not mandatory config. This is required only for classification networks

2. Please use to performance simulation on these networks - "http://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/latest/exports/docs/tidl_j7_01_01_00_10/ti_dl/docs/user_guide_html/md_tidl_naps_executioninfo.html"

3. Yes, user can run the Caffe-based SSD network on TDA4 refer below for a list of validated nets

http://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/latest/exports/docs/tidl_j7_01_01_00_10/ti_dl/docs/user_guide_html/md_tidl_models_info.html

4. Optimization of Sparse models are not supported in TDA4

0 WEI CHIH CHERN over 5 years ago in reply to kumar.desappan

Intellectual 320 points

kumar.desappan said:

4. Optimization of Sparse models are not supported in TDA4

Isn't the sparse optimization done during training rather than run time, or is it? If I had a sparsed model, does it have no speed improvement at all or simply TDA4 cannot run sparsed SSD model?

Thank you.

0 kumar.desappan over 5 years ago in reply to WEI CHIH CHERN

TI__Mastermind 22145 points

TDA4 can run sparsed mode, But speed improvement is not expected.

0 WEI CHIH CHERN over 5 years ago in reply to kumar.desappan

Intellectual 320 points

Hi Kumar,

Another question is that, quantization optimization is done during runtime when running the model on the TDA2x board right? If so, does the given infer_video_object.py uses quantization when perform detection? Thanks.

0 Manu Mathew over 5 years ago in reply to WEI CHIH CHERN

TI__Genius 11506 points

infer_video_object.py does not use quantization when performing detection. But when you run it on TDA2, quantization will be performed.

You can read more about Deep Learning training software that we provide for TDA4 here: https://github.com/TexasInstruments/jacinto-ai-devkit

Best regards,

Manu

0 WEI CHIH CHERN over 5 years ago in reply to Manu Mathew

Intellectual 320 points

Quantization is done for the purpose of speed, but it kind of downgrading my model's mAP quite a lot. Is the impact inevitable? Or any way to reduce the impact?

I noticed that Pytorch implementation used quantization aware training, does this allows lower performance impact than TDA2's runtime quantization?

0 Manu Mathew over 5 years ago in reply to WEI CHIH CHERN

TI__Genius 11506 points

Hi,

Quantization scheme in TIDL for TDA2 is rather simple - so certain networks can have accuracy degradation. The accuracy degradation may be more pronounced for networks that use Depthwise convolution layers. So its better to avoid it if you are using TIDL for TDA2.

Most of the development effort is now focused on TDA4 these days.

As you noticed tools for QAT are provided in PyTorch jacinto-ai-devkit for TDA4. Also TIDL for TDA4 will have Advanced calibration options in the next release that should help to reduce much of quantization accuracy loss even without using QAT.

So, since most of our development is now focused on TDA4, you have more options and support available on TDA4. So if you are considering TDA4, I would certainly encouraging it.

Introduction: https://www.ti.com/product/TDA4VM

If you read the datasheet: https://www.ti.com/lit/ds/symlink/tda4vm.pdf

It says, it has 8 TOPS (8 Tera Operations per second, 8bit) for Deep Learning. That is much much more compared to TDA2.

Best regards,

0 WEI CHIH CHERN over 5 years ago in reply to Manu Mathew

Intellectual 320 points

Hi,

Thanks for the thorough replies, Manu and Kumar. It's been really helpful.

Best regards,

Wei Chih

0 WEI CHIH CHERN over 5 years ago in reply to kumar.desappan

Intellectual 320 points

kumar.desappan said:

1. Are you referring to import config "resize_width & resize_height" if yes, this not mandatory config. This is required only for classification networks

After tweaking a bit, I discovered some issues when training SSD for detection:

1. Changing ssd_size from '768x320' to '300x300' has no impact on '...../initial/deploy.prototxt', input_shape in deploy.prototxt still shows:

input_shape {
dim: 1
dim: 3
dim: 768
dim: 320
}

and ssd_size '768x320' & '300x300' took same amount of GPU memory with same batch size during training process.

2. setting forced_color=0 also has no impact on ....../initial//deploy.prototxt. input_shape still shows input as 3 channels images:

input_shape {
dim: 1
dim: 3 // forced_color=0
dim: 768
dim: 320
}

Do we need to manually convert all our training/testing/validation data to grayscale?

Thank you,

Wei Chih

0 Manu Mathew over 5 years ago in reply to WEI CHIH CHERN

TI__Genius 11506 points

I have just assigned this thread to the expert.

0 Soyeb Nagori over 5 years ago in reply to Manu Mathew

TI__Intellectual 830 points

Can you pls tell me where are you making changes to change the input resolution? Also can you pls share your training log file?
Meaning of force_color=True is when during training it encounters gray scale image it will convert to color so you don't need to do anything extra.

0 WEI CHIH CHERN over 5 years ago in reply to Soyeb Nagori

Intellectual 320 points

Hello Najori,

1. In the 'train_image_object_detection.sh', there are parameters such as ssd_size & resize_width & resize_height:

I set,

dataset=ti-custom-cfg1

ssd_size='512x512' #this doesn't change input_shape in generated /initial/deploy.prototxt

elif [ $dataset = "ti-custom-cfg1" ]
then

resize_width=768 # these will change input_shape in generated /initial/deploy.prototxt
resize_height=320 # these will change input_shape in generated /initial/deploy.prototxt
crop_width=768
crop_height=320

Training log file (I've git clone caffe-jacinto-models again, ssd_size still have no control on input_shape in deploy.prototxt):

8484.run.log

0 Soyeb Nagori over 5 years ago in reply to WEI CHIH CHERN

TI__Intellectual 830 points

This is expected behavior. To actually change network input size the followings need to be set appropriately.

resize_width=768
resize_height=320
crop_width=768
crop_height=320

ssd_size is used for something else. May be it should have been named differently.

Processors

Processors forum

TDA2E: Different SSD models & some TDA4 questions