AM5728: All frames are dropped when running TIDL usecase with camera

Hi Gary,

>> I tested the model worked fine on PC, but didn't work on TDA2x.

Can you check if you get proper output from TIDL import tool itself. 

FYI, the import tool also runs TIDL inference on PC and will give output detection's, so first check are you getting proper output in the import itself then you can run the model on TDA2x.

Thanks,

Praveen   

Hi Praveen,

>> ...check if you get proper output from TIDL import tool itself.

>> the import tool also runs TIDL inference on PC and will give output detection's...

How to check from TIDL import tool itself?

Are there any reference documents?

Thank you,

Gary

Hi Gary,

The import tool will dump the final output in "stats_tool_out.bin" file along with layer level traces (outputs) for each layer.

Please use markBox.c file from below e2e thread to visualize the import tool output and if you get correct output here then you can run your model on VSDK.
https://e2e.ti.com/support/processors/f/791/t/679186

Thanks,

Praveen

Hi Praveen,

I used jpeg image as a input of markBox.c.

However, the output PPM file looks like noise, which as attachment shows.

So my questions are,

1. How to use jpeg format image file as an input of markBox.c?

     Should I convert the jpeg file to .y format?

     If so, how can I generate a '.y' file

thank you,

Gary

Hi Gary,

You need to use ".y" file as an input in the markBox.c file. 

You will get this "y" file as part of pre-processing in the import tool and it will be dumped as "trace_dump_0_768x320.y" in the import tool traces. This is nothing but the ".y" conversion of the input jpeg image, so use this "trace_dump_0_768x320.y" file in the markBox.c file.

Thanks,

Praveen 

Hi Praveen,

I used the dump .y file "trace_dump_0_512x512.y" in the markBox.c, but the result is the same.

So here I list my steps to make sure there is no misunderstanding.

My jpg file name is 000019.jpg, which is a color image.

1. I changed the "sampleInData" field in import tool setting file to "\path\to\000019.jpg"

2. Running import tool, I got dump files. One of these is "trace_dump_0_512x512.y"

3. I used "trace_dump_0_512x512.y" in the markBox.c to get PPM file

Are there any problems for these steps?

Thank you,

Gary

Hi Gary,

Can confirm that you are taken care on below points,  

(1) Did you update the markBox.c file for your image resolution ?

(2) Are you reading the "tats_tool_out_eve.bin" file in binary mode? as there is an issue in opening this file in the markBox.c as explained in the thread 

      fb = fopen("stats_tool_out_eve.bin", "rb");

(3) Did you visualize the "trace_dump_0_512x512.y" file ? it should look similar to the input jpeg file.

After checking all the above points still if you see the issue, please attach your import config file for review.

Thanks,

Praveen

Hi Praveen,

>>(1) Did you update the markBox.c file for your image resolution ?

I changed orig size as my image size.

>> (2) Are you reading the "tats_tool_out_eve.bin" file in binary mode? as there is an issue in opening this file in the markBox.c as explained in the thread

I don't have "stats_tool_out_eve.bin", so I used "stats_tool_out.bin" instead.

I have tried the parameter "r" & "rb", the results are the same.

>>(3) Did you visualize the "trace_dump_0_512x512.y" file ? it should look similar to the input jpeg file.

Yes, it is same as my original input image.

So here are my import files, thank your for your help.

Gary

8561.importFiles.zip

Hi Gary, 

From the attached files, I see that you have not taken care of 1 and 3.

>> (1) I changed orig size as my image size.

You should replace all the values (768 and 320) with 512 accordingly  

>> (3) Yes, it is same as my original input image. 

I see that "trace_dump_0_512x512.y" file is some corrupted, this may because of  "rawSampleInData = 1" in import config file, but your input is not raw data, so change this value "rawSampleInData = 0" and then try importing 

Thanks,

Praveen 

Hi Praveen ,

1. After changing "rawSampleInData = 0", the result is still not correct.

2. I have also tested another image for comparing the detected results between original trained model and model transferred by import tool. The results are listing below.

OD model (Original)

OD model (import tool)

It  looks like something wrong after transferring by import tool.

My import file is same as previous one.

Are there anything wrong with my import settings?

Can you suggest me how to solve this problem?

Thank you,

Gary

Hi Gary,

>>>

OD model (import tool)

>> It  looks like something wrong after transferring by import tool.

Is this output with TI model or with your own model ?

Thanks,

Praveen

Hi Praveen,

>>Is this output with TI model or with your own model ?

Both two results are coming from my own model.

Thanks,

Gary

Hi Gary,

I tried importing your model shared in the previous post. I see that in your model, for some of the layers weights are not normalized in the training. These weights are causing issue in the import process.

Can you try to do L1 and L2 regularization on both BatchNorm and conv layers in your model while training. 

Thanks,

Praveen

Hi Praveen,

>> Can you try to do L1 and L2 regularization on both BatchNorm and conv layers in your model...

I checked my solver file and I have already set the regularization_type = "L1".

Is this same as above which you suggest me to do?

If not, can you give me some hints how to do this? I have no idea how to add regularization in both BatchNorm and conv layers.

Here is my solver file 

train_net:"models/VGGNet/jdetnet/train_jdetnet.prototxt"
test_net: "models/VGGNet/jdetnet/test_jdetnet.prototxt"
test_iter: 2
test_interval: 10
base_lr: 0.001
display: 100
max_iter: 500
lr_policy: "poly"
gamma: 0.1
power: 4.0
momentum: 0.9
weight_decay: 1e-05
snapshot: 2000
snapshot_prefix: "models/VGGNet/jdetnet/jdetnet"
solver_mode: GPU
device_id: 0
random_seed: 33
debug_info: false
snapshot_after_train: true
regularization_type: "L1"
test_initialization: true
average_loss: 10
stepvalue: 30000
stepvalue: 45000
iter_size: 4
type: "SGD"
display_sparsity: 2000
sparse_mode: SPARSE_UPDATE
sparsity_target: 0.7
sparsity_step_factor: 0.05
sparsity_step_iter: 2000
sparsity_start_iter: 0
sparsity_start_factor: 0.25
sparsity_threshold_maxratio: 0.2
sparsity_itr_increment_bfr_applying: true
sparsity_threshold_value_max: 0.2
eval_type: "detection"
ap_version: "11point"
show_per_class_result: true

Thank you,

Gary

Hi Praveen,

Have you tried to convert the object detection model from caffe-jacinto-models-caffe-0.16 through import tool?
Does the test result correct on your side?

I did two tests by using the raw object detection model from caffe-jacinto-models-caffe-0.16.
The tested models are l1reg and sparse located in "/JDetNet/ssd512x512_ds_PSP_dsFac_32_fc_0_hdDS8_1_kerMbox_3_1stHdSameOpCh_1/".

The first one is tested on host emulation by using trace_dump_0_768x320.y.txt .
I ran the steps you mentioned in the thread and visualized through markBox.c.
The result is as follow.
markedBoxes.ppm.txt

The other test is running on TDA2 with a video clip.
However, the result is same as using my own trained model which is mentioned in my previous post
( https://e2e.ti.com/support/processors/f/791/t/839810 ).

Here are my questions,
1. If you have tried the raw model and tested it successfully, can you provide or point me out the related files?
   Files such as, deploy.proto, caffemodel, tidl_import_XXX.txt, etc.
   
2. In this thread( https://e2e.ti.com/support/processors/f/791/t/679186 ), I can't get the same result as you do.

   So, I would like to know which model do you use?
   Can you also share the related files or are there any documents for reference?
    
   
   Thank you,
   Gary