TDA4VM: [TIDL] precision problem when using YuvFormat = 0

Hi Henry,

1. No, accuracy problem is not normal

2. Did you change anything in the "tidl_import_mobileNetv2_yuv.txt" file? if so please share the import config file ?

3. inFileFormat = 2 is supported for compressed images list

Thanks,

Praveen

Hi,Praveen

My sdk version is 07_02_00_06, configuration file is as follows
--------------------------------- tidl_import_mobileNetv2_yuv.txt ---------------------------------
modelType = 1
numParamBits = 8
quantizationStyle = 2
inputNetFile = "../../test/testvecs/models/public/tensorflow/mobileNet_v2/mobilenet_v2_1.0_224_final.pb"
outputNetFile = "../../test/testvecs/config/tidl_models/tensorflow/tidl_net_mobilenet_v2_1.0_224_yuv.bin"
outputParamsFile = "../../test/testvecs/config/tidl_models/tensorflow/tidl_net_mobilenet_v2_1.0_224_yuv_"
inDataNorm = 1
inMean = 128 128 128
inScale = 0.0078125 0.0078125 0.0078125
inWidth = 224
inHeight = 224
inNumChannels = 1
inData = "../../test/testvecs/input/airshow_yuv420_224x224_224x112.yuv"
perfSimConfig = "../../test/testvecs/config/import/device_config.cfg"
postProcType = 1
inFileFormat = 1
rawDataInElementType = 0
foldPreBnConv2D = 1
inYuvFormat = 0
numFrames = 1
compileConstraintsFlag = 579
--------------------------------- tidl_infer_mobileNetv2_yuv.txt ---------------------------------
inFileFormat = 1
postProcType = 1
numFrames = 1
netBinFile = "testvecs/config/tidl_models/tensorflow/tidl_net_mobilenet_v2_1.0_224_yuv.bin"
ioConfigFile = "testvecs/config/tidl_models/tensorflow/tidl_net_mobilenet_v2_1.0_224_yuv_1.bin"
inData = "testvecs/input/airshow_yuv420_224x224_224x112.yuv"
outData = "testvecs/output/airshow_mobilenetv2_tf_yuv.bin"
debugTraceLevel = 0
writeTraceLevel = 3
#flowCtrl = 12

I looked at the airshow_mobilenetv2_tf_yuv.bin file in the output folder, the label is 658. The result should be 896, right?
Best Regards,
Henry

Hi Henry,

Yes, the result is not 896 because we are running only image, you will not get 896 for even "tidl_infer_mobileNetv2.txt", as there is no such accuracy issue for yuv input.

You need to run more images to get actual accuracy for mobileNetv2 with yuv input. For running multiple inputs you can concat them and give via inFileFormat = 1

Thanks,

Praveen

Hi,Praveen

      Thank you for your reply.

     Can you provide me the configuration file for mobileNetv2 with yuv input to get the actual accuracy? 

Best Regards,
Henry

Hi Henry,

In the import config, set calibrationOption = 7 and use at least twenty images then try.

Thanks,

Praveen

Hi,Praveen
In the import file for running multiple inputs，I connected multiple input pictures into a yuv file.
Is this configuration sufficient? Do I need to set the numframes parameter?
I set writeTraceLevel = 3 to get intermediate calculation results.The data processed by the ColorConversion layer does not look normal.
What is the conversion process from YUV to RGB in TIDL? What should I do to determine if the result is correct?
Best Regards,
Henry

Hi Henry,

>>  Do I need to set the numframes parameter?

Yes, you need to set numframes parameter as this is not directly drivable from the yuv file.

>> What is the conversion process from YUV to RGB in TIDL? What should I do to determine if the result is correct?

You should check output of layer number 3, then you can see correct converted output.

Thanks,

Praveen

Hi,Praveen

I compared these two binary files, tidl_infer_mobileNetv2_yuv.txt_0003_00003_00224x00224.y and tidl_infer_mobileNetv2.txt_0000_00003_00224x00224.y.

In the tidl_infer_mobileNetv2_yuv.txt_0003_00003_00224x00224.y file, the data of the last two channels are not converted to RGB.

I don't know the reason for the problem.

It would be a great help if you can provide me with import_mobileNetv2_yuv and infer_mobileNetv2_yuv files.

Thanks,

Henry

Hi Henry,

I hope you had provided input in the "YUV420 NV12" format, because other formats are not supported. In that case layer0 should have only one channel Y and layer1 should be UV component. Layer2 is the conversion layer and Layer 3 is the concat layer so layer 3 should be the correct correct. 

Starting of the import log should look like this below,

0|TIDL_DataLayer |input | 0| -1| 1| x x x x x x x x | 0 | 0 0 0 0 | 1 1 224 224 | 0 |
1|TIDL_DataLayer |Input_UVSemiPlanar_0 | 0| -1| 1| x x x x x x x x | 1 | 0 0 0 0 | 1 1 112 224 | 0 |
2|TIDL_ColorConversionLayer |Convert_UV420SP_To_Planar0 | 0| 1| 1| 1 x x x x x x x | 2 | 1 1 112 224 | 1 2 224 224 | 100352 |
3|TIDL_ConcatLayer |Concat_Y_UV_0_original | 0| 2| 1| 0 2 x x x x x x | 3 | 1 1 224 224 | 1 3 224 224 | 150528 |
4|TIDL_ConvolutionLayer |MobilenetV2/Conv/Relu6 | 0| 1| 1| 3 x x x x x x x | 4 | 1 3 224 224 | 1 32 112 112 | 11640832 |
5|TIDL_ConvolutionLayer |MobilenetV2/expanded_conv/depthwise/Relu6 | 0| 1| 1| 4 x x x x x x x | 5 | 1 32 112 112 | 1 32 112 112 | 4415488 |

There are no other changes the import config file except for calibrationOption = 7 and numframes.

Thanks,

Praveen

Hi,Praveen

I checked the import log, which is the same as your log. The yuv file I used is testvecs/input/airshow_yuv420_224x224_224x112.yuv.

The data for Layer 3 looks like this.

The first channel is the Y channel in 16bit format. Next is the 8bit UV channel.This looks like the conversion was unsuccessful

Best Regards,

Henry

Hi Henry,

I have attached the import and infer config files and layer level trace dumps with yuv input. Please check if can match them at your end.

Please note that I am latest release so trace file name may look different, you can ignore that and still compare the outputs.

e2e_yuvInput.zip

Thanks,

Praveen

Hi,Praveen

Thank you for the files. Use numParamBits = 8 and the result is correct. The previous question was because I used numParamBits = 16 in the import file.

Now I use 16bit yuv for input and I can get the expected result on PC linux. But the result on the device is not good.

I modified the vision_apps/apps/dl_demos/app_tidl program and used 16bit yuv420 as input.

By comparing the intermediate files on the PC linux and the device, I found that the layer0 and layer1 data are correct. But layer 2 is quite different.

The data of layer2 on PC Linux is 16bit UV data. On the device looks abnormal like the following.

When numParamBits = 16, is yuv to rgb supported? Why is the result bad on the device?

Best Regards,

Henry

Hi Henry,

The result is bad because we don't support yuv to rgb conversion when numParamBits = 16. We have support for only 8-bit yuv input.

Thanks,

Praveen

Hi,Praveen

When importing with numParamBits = 16, my model has better accuracy.

Will TIDL support yuv to rgb conversion when numParamBits = 16?

Best Regards,

Henry