AM5749: How can I compare the traces between desktop caffe and tidl stats tool?

user4652979

Part Number: AM5749

The inference result is different between desktop caffe and am5749. And I have change the number of bits for weights, but the result is still different. The guide said I should compare the intermediate results. And the stats tool generates some trace_*.y file and .bin file. Now, how can I get the intermediate results from caffe-jacinto and find the difference? And what's the trace_*.file?

Another question, if I have a single channle .bmp file, is it right to convert the .bmp file to .y file by following cmd?

convert code.bmp -separate +channel -swap 0,2 -combine -colorspace sRGB ./test_bgr.bmp
convert ./test_bgr.bmp -interplace plane BGR:SAMPLE_IMG.y

over 5 years ago

0 user4652979 over 5 years ago

Expert 1860 points

Can somebody help me? My project is very urgent.

0 Paula Carrillo over 5 years ago in reply to user4652979

TI__Mastermind 38990 points

Hi Sorry for the delay, will take a look and come back to you soon. With respect to format conversion, I haven't used your system, but, at a simple look seems a correct way to create NCHW (BGR - plane interleaved) which is the format expected by TIDL.

thank you,

Paula

0 user4652979 over 5 years ago in reply to Paula Carrillo

Expert 1860 points

Thanks for your reply and I will wait for your help.

0 Paula Carrillo over 5 years ago in reply to user4652979

TI__Mastermind 38990 points

Hi, could you elaborate how are you comparing the inference results between Caffe and AM57x TIDL? is there any accuracy dropped? if so how much?. Also, just to confirm, are you using the same trained Caffe model when comparing (ex: using same sparsing, batch size etc)? By the way which model are you using? Mobilenet? (just to know)

Thank you,

Paula

0 user4652979 over 5 years ago in reply to Paula Carrillo

Expert 1860 points

Thanks for your reply.

I use ssd-jacinto model. I trained the model on computer and import the model by the tool which named tid_model_import.out. When I run the model on AM5749 on same image, the ssd box position will shift 10%~30% to the right and down compared to the result of running on the computer.

0 Paula Carrillo over 5 years ago in reply to user4652979

TI__Mastermind 38990 points

Hi thank you for clarifying. Quick question it is always a shift to the right/down? are you using camera input or some images? any way I can reproduce it here?..

Paula

0 user4652979 over 5 years ago in reply to Paula Carrillo

Expert 1860 points

I used camera input and save the input to .bmp to retest it on the computer caffe-jacinto. And you said reproduce, which do you need ?

0 Paula Carrillo over 5 years ago in reply to user4652979

TI__Mastermind 38990 points

Hi, if you use individual images, do you still see the same type of offset/difference between your PC's Caffe and AM57x inference?. Wondering if there is any frame sync issue with video input.

For reproducing I would need to have a model that shows the issue.. if you cannot share your model, no issues, we will figure it out another way.

thank you,

Paula

0 user4652979 over 5 years ago in reply to Paula Carrillo

Expert 1860 points

It's not the sync issue and I have confirmed.

I can give you the model and can you give me your email?

0 Paula Carrillo over 5 years ago in reply to user4652979

TI__Mastermind 38990 points

Hi Ok I will send it in a private message. By the way it is always shift from top-right? .. if so, that would still make suspect is could be related with image pre-processing/post-processing. Maybe we need to check how the image is resize/crop before is overlay with the boxes...

thank you,

Paula