TDA4VM: Unexpected Pre-proc node output

Seetharama Raju

Part Number: TDA4VM

Tool/software:

Hi sir,

I'm working on an application with pipeline scaler->pre-proc->TIDL->post-proc. SDK is 08.02.00.05, target is TDA4VM/J721E.

My DL model is 16 bit. So pre-proc output should give 16 bit RGB tensor.

I dumped the pre-proc output and checked it using VOOYA application. It is expected that pre-proc output should have border padding of 1 1 0 2 (left top right bottom). So I found the tensor size is 3x225x227. (Model size 224x224)

But the values of padding( R G B) are in the range of 65535 ( 2^16).

I dumped the pre-proc output for demo app_tidl_16bit segmentation app, and checked for the same.The values of padding( R G B) are 0 in this case.

The pre-proc dumps can be seen below. Left My application , right SDK demo seg(16bit)

Note:

->Im getting invalid output from the application, I'm assuming this could be the reason.I validated custom post-processing node and it is working fine. So either pre-proc or TIDL must be causing problem.

->I got valid and expected output from the demo segmentation app.

-> I even verified with another model of szie 512x320 (16 bit), I faced the same issue for this also.

Please guide me regarding this.

Thanks,

Raju.

10 months ago

0 Seetharama Raju 10 months ago

Prodigy 170 points

Hello sir,

Could you please respond!

Thanks,

Raju.

0 Wen Li 9 months ago in reply to Seetharama Raju

TI__Expert 7891 points

Hi Raju; In order to understand your application and question better, could you show us your pre-processing code/process? Including the dumping pre-proc data and how you were using VOOYA to check it.

Please provide your model as well if you can.

Thanks and regards

Wen Li

0 Seetharama Raju 9 months ago in reply to Wen Li

Prodigy 170 points

Hi sir,

Wen Li said:
could you show us your pre-processing code/process?

The pre-processing node used is from SDK only. i.e tivxImgPreProcNode (dsp-c66 core)

Wen Li said:
Including the dumping pre-proc data

I'm using the API writePreProcOutput(); availble in app_pre_proc_module.c. only difference in the code is "uint16_t" in the call "fwrite(data_ptr, sizeof(uint16_t), tensor_strides[0] * tensor_sizes[0] * tensor_sizes[1] * tensor_sizes[2], fp), because pre-proc output should be uint16_t.

Wen Li said:
how you were using VOOYA to check it

in VOOYA app, we can keep file format as image size 225x227, colour space: RGB, Data container: Planar 4:4:4, channel order rgb, bit depth 9 bit.

Wen Li said:
Please provide your model as well if you can

Pardon me that we cannot share the model for confidentiality purpose.

The main issue is when model is used for stand-alone target inference, the output from the model is fed to the custom-post processing node, it is giving us expected output. But model is giving different output when used in the application resulting into wrong output.

I have dumped APP_TIDL_TRACE_DUMP in the app and enabling writetracelevel to 1 in the standalone inference. Input to the TIDL is found to be different in both cases for the same input image.

Please guide me regarding the issue.

Thanks,

Raju.

0 Seetharama Raju 9 months ago in reply to Seetharama Raju

Prodigy 170 points

Hi sir,

Any update regarding this?

Thanks,

Raju.

0 Wen Li 9 months ago in reply to Seetharama Raju

TI__Expert 7891 points

Hi; Since you are using the 16bit, So the pixel value of RGB color components is in the range of 0 to 65535 (2^16 -1). Do you see anything wrong with that (during pre-processing, in term of pixel value for padding)?

I try to understand what is your key concern/question.

Thanks and regards

Wen

0 Seetharama Raju 9 months ago in reply to Wen Li

Prodigy 170 points

Hi sir,

Wen Li said:
Since you are using the 16bit, So the pixel value of RGB color components is in the range of 0 to 65535 (2^16 -1). Do you see anything wrong with that (during pre-processing, in term of pixel value for padding)?

Eventhough pixel values are supposed to be in the range of 0 to 65535 (2^16 -1), those are in the range of 0 to 255 only.( we know this behaviour is due to the algorithm at $RTOS/tiadalg/tiadalg_image_preprocessing/alg/tiadalg_image_preprocessing_c66.c or tiadalg_image_preprocessing_cn.c).

The main problem can be simplified as " difference in input image to TIDL in standalone inference and application for the same input image to the pipeline".

For an instance, I have shared below the two files. The two files are pre-processing outputs( input to the TIDL). We can find the difference in the values for the same input image. The first file is standalone( correct case) and the second one is from the application ( Wrong case)

stdalone_tidl_input_firstimage.txt

app_tidl_input_firstimage.txt

Due to difference in the values/pixels, subsequent layers in the DL network are giving wrong output in case of the application.

Please guide me regarding this.

Thanks,

Raju

0 Seetharama Raju 9 months ago in reply to Seetharama Raju

Prodigy 170 points

Hi sir,

Any update regarding the above issue?

Thanks,

Raju

0 Wen Li 9 months ago in reply to Seetharama Raju

TI__Expert 7891 points

Hello;

What do you mean by "Eventhough pixel values are supposed to be in the range of 0 to 65535 (2^16 -1), those are in the range of 0 to 255 only"

Have you done any decimation (filtering) in your preprocessing? I am just guessing, since I could not see your code.

I have looked at your two *.txt files. They both are just bunch of numerical data in 8bit format. It seems like you using 8bit RGB..

Since you could not send the model, that is ok, but we would have to assume the model stays the same. Can you compare your pre-processing part of the standalone vs pre-processing part of your application?

Thanks and regards

Wen Li

0 Seetharama Raju 9 months ago in reply to Wen Li

Prodigy 170 points

Hi sir,

Wen Li said:
Have you done any decimation (filtering) in your preprocessing? I am just guessing, since I could not see your code

The pre-processing node used is from SDK only. i.e tivxImgPreProcNode (dsp-c66 core). I didnt do any decimation.

Wen Li said:
but we would have to assume the model stays the same.

Yes, the model is same.

Wen Li said:
Can you compare your pre-processing part of the standalone vs pre-processing part of your application?

As mentioned application preproc is tivxImgPreProcNode and we know that standalone preprocessing is done by tidl_model_import.out binary.

I have uploaded below the import configuration file used.

Fullscreen import_file.txt Download

# Set modelType = 2 for onnx models
modelType          = 2

# Set numParamBits as 8 for INT8 quantization and 16 for INT16 quantization
numParamBits       = 16
numFeatureBits     = 16

# Set quantizationStyle = 2 for Linear mode and 3 for power of 2 scale quantization
quantizationStyle = 3

# Give path of onnx model in inputNetFile
inputNetFile = "/home/raju/TI_SDK/08.02.00.05/ti-processor-sdk-rtos-j721e-evm-08_02_00_05/tidl_j721e_08_02_00_11/ti_dl/test/testvecs/models/public/onnx/data/model.onnx"

# Give path where TIDL net model is to be saved in outputNetFile
outputNetFile     = "/home/raju/TI_SDK/08.02.00.05/ti-processor-sdk-rtos-j721e-evm-08_02_00_05/tidl_j721e_08_02_00_11/ti_dl/test/testvecs/models/public/onnx/data/output/out.bin"

# Give path where TIDL IO bin is to be saved in outputParamsFile
outputParamsFile   = "/home/raju/TI_SDK/08.02.00.05/ti-processor-sdk-rtos-j721e-evm-08_02_00_05/tidl_j721e_08_02_00_11/ti_dl/test/testvecs/models/public/onnx/data/output/out_io"

outDataNamesList = "out,signals"

# Set inDataNorm =1 to if normalized data is given to model
inDataNorm  = 1

# Mean value need to be subtracted for each channel of all input tensors. Only applicable if inDataNorm = 1
inMean = 0 0 0

# Scale value needs to be multiplied after mean subtract for each channel of all input tensors. Only applicable when inDataNorm = 1
inScale = 0.003921568627 0.003921568627 0.003921568627

# Set inWidth as input image width
inWidth  = 224

# Set inHeight as input image height
inHeight = 224

#inResizeType = 0

#resizeWidth = 256

#resizeHeight = 256

# Set number of channels of the input image
inNumChannels = 3

# Set number of frames given for calibration
numFrames = 50

# Set path of txt file containing path of images used for calibration
inData = "/home/raju/TI_SDK/08.02.00.05/ti-processor-sdk-rtos-j721e-evm-08_02_00_05/tidl_j721e_08_02_00_11/ti_dl/test/testvecs/models/public/onnx/data/calib.txt"

# Set inDataFormat = 1 if input is RGB format or 0 if BGR format
inDataFormat = 1

inFileFormat = 2

#debugTraceLevel = 1
#writeTraceLevel = 3

# Set path of Network Compiler Configuration file. You may give the device_config path
perfSimConfig = /home/raju/TI_SDK/08.02.00.05/ti-processor-sdk-rtos-j721e-evm-08_02_00_05/tidl_j721e_08_02_00_11/tidl_tools/device_config.cfg
perfSimTool = /home/raju/TI_SDK/08.02.00.05/ti-processor-sdk-rtos-j721e-evm-08_02_00_05/tidl_j721e_08_02_00_11/tidl_tools/ti_cnnperfsim.out
graphVizTool = /home/raju/TI_SDK/08.02.00.05/ti-processor-sdk-rtos-j721e-evm-08_02_00_05/tidl_j721e_08_02_00_11/tidl_tools/tidl_graphVisualiser.out

tidlStatsTool = /home/raju/TI_SDK/08.02.00.05/ti-processor-sdk-rtos-j721e-evm-08_02_00_05/tidl_j721e_08_02_00_11/tidl_tools/PC_dsp_test_dl_algo.out

Please guide me!

Thanks,

Raju.

0 Seetharama Raju 8 months ago in reply to Seetharama Raju

Prodigy 170 points

Hi sir,

Any update based on the above information?

Thanks,

Raju.

0 Wen Li 8 months ago in reply to Seetharama Raju

TI__Expert 7891 points

Hi;

We still could not understand your problem completely. So let me re-group:

1. Are you saying padding data are wrong? they are out of range? By read your post, it is within the 16bit range. Why do you think it is wrong.

2. You used VOOYA to check the images/video, but results are wrong. What will be the right definition. VOOYA is just a raw video player.

3. Why do you say you are getting invalid output. What is your invalid output look like?

4. "I even verified with another model of szie 512x320 (16 bit), I faced the same issue for this also" what is the issue? could you define the issue

Could you describe your problem/issue clearly? Since we don't have your source code and architecture of your application.

Thanks and regards

Wen Li

0 Seetharama Raju 8 months ago in reply to Wen Li

Prodigy 170 points

Hi sir,

Wen Li said:
Are you saying padding data are wrong? they are out of range?

The padding issue is resolved. The border padding values are 0. i.e left, top, right and bottom. The same behaviour is observed for the demo applications. I request you to ignore the padding issue I raised at the starting query of the thread.

Wen Li said:
Could you describe your problem/issue clearly? Since we don't have your source code and architecture of your application.

My pipeline is File_input-'>scaler->pre-proc->TIDL->post-proc->output

When i give an input image, output from the application is not proper. It shoud give one of the vision fail classes. It is giving a wrong class.

As part of debugging,I have done standalone target inference with the same input image thenTIDL inference output is obtained. I passed this output to post-processing node in the application, then it gave the exact class/output. in this way post-proc node is found not the bottleneck.

Scaler node is just used for down scaling,so it also cannot be causing the issue. As as a result of debuuging the following observation is made.

Seetharama Raju said:
The main problem can be simplified as " difference in input image to TIDL in standalone inference and application for the same input image to the pipeline".

For an instance, I have shared below the two files. The two files are pre-processing outputs( input to the TIDL). We can find the difference in the values for the same input image. The first file is standalone( correct case) and the second one is from the application ( Wrong case)

stdalone_tidl_input_firstimage.txt

app_tidl_input_firstimage.txt

Due to difference in the values/pixels, subsequent layers in the DL network are giving wrong output in case of the application.

I hope I made the problem statement clear. I want your support in deubbing the issue , how can we resolve it?

Thanks a lot,

Raju.

Processors

Processors forum

TDA4VM: Unexpected Pre-proc node output