CCS: TDA4x:: Trace dumps issues in "app_tidl_od"

Hi Vyom,

The trace data buffer size is defined in app_tild_module.c.

Dear Sir,

My model has a total 127 layers, for which I need the layer dumps of 

Layer 109, 120,127 for post-processing.

I need the layer dumps in float.bin.

Can you please help me to overcome this issue as per the model requirements.

Thanks and Regards,

Vyom Mishra

Dear Sir,

I have made changes to

#define TIVX_TIDL_TRACE_DATA_SIZE  (128 * 1024 * 1024 * 13)

And I didn't face linker issues on PC_Emulation build.

Now All 127 traces are dumped.

I need your help to disable the ".y" dumps.

Is it possible to do that, if Yes kindly help me.

For your reference,

my name:/home/sithara/ti/j8/psdk_rtos_auto_j7_06_02_00_21/vision_apps/app_tidl_od_out/0000000001_0000_00003_00608x00608.y|||||||||||||my size: 1108992
my name:/home/sithara/ti/j8/psdk_rtos_auto_j7_06_02_00_21/vision_apps/app_tidl_od_out/0000000001_0000_00003_00608x00608_float.bin|||||||||||||my size: 4435968
my name:/home/sithara/ti/j8/psdk_rtos_auto_j7_06_02_00_21/vision_apps/app_tidl_od_out/0000000001_0001_00003_00602x00602.y|||||||||||||my size: 1087212
my name:/home/sithara/ti/j8/psdk_rtos_auto_j7_06_02_00_21/vision_apps/app_tidl_od_out/0000000001_0001_00003_00602x00602_float.bin|||||||||||||my size: 4348848
my name:/home/sithara/ti/j8/psdk_rtos_auto_j7_06_02_00_21/vision_apps/app_tidl_od_out/0000000001_0002_00016_00602x00602.y|||||||||||||my size: 5798464
my name:/home/sithara/ti/j8/psdk_rtos_auto_j7_06_02_00_21/vision_apps/app_tidl_od_out/0000000001_0002_00016_00602x00602_float.bin|||||||||||||my size: 23193856

Thanks and Regards,

Vyom Mishra

Hi Vyom,

  Why do you need layer dumps for post-processing? Aren't these layers output of your network?

Regards,

Anshu

Dear Sir,

The model we are working with has no Post-processing enabled.

So, we need to have the feature dumps from the layers(109,120,127) to process them with our Post-Processing script.

For  app_tidl_od,

I need to access those buffers which hold these three feature dumps for each frame to process it to Post-processing Kernel written by us.

For this extraction, I have planned to enable traces and obtain the float values of each required feature dumps from the buffer.

If you can help us to extract these specific features dumps from another source of code then please let me know the part of code.

I will be thankful for this.

Thanks and Regards,

Vyom Mishra

Hi Vyom,

   You should declare these layers  ( the ones you need for post processing) as output of the network and in this case TIDL will return these layers as output. The trace dumps is supposed to be used for debugging purpose and is not intended to get the outputs from the network. If your model is not a caffe based model then you can use outDataNamesList to list all the outputs of the network. Refer below link to get more details of this vairable

http://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/latest/exports/docs/tidl_j7_01_01_00_10/ti_dl/docs/user_guide_html/md_tidl_model_import.html

Regards,

Anshu

Dear Sir,

Thanks for the reply.

I have used this parameter " outDataNamesList" for importing the model and provided all three names of the layer to the list.

 "in this case, TIDL will return these layers as output"

By the above statement you mean to say that all three layers output will be stored in the "output.bin" that is the model output stored in the binary file.

1. If it is the case, then will it be writing the output of the layers(i.e., float values) in the contagious form to the binary file like (layer109,layer120,layer127).bin.      Kindly correct me with the understanding of the output dumps of the model.

2. Please quote the portion of the code in the application where I can get this tidl model output

As per my understanding, in the App

app_create_graph_draw_detections(obj->graph, &obj->drawDetectionsObj, obj->tidlObj.output_tensor_arr[0], obj->scalerObj.output2.arr[0]);

Here  "obj->tidlObj.output_tensor_arr[0] " has the labels for the frame(if post-processing is enabled). Is it possible to access the tidl model output with the feature dumps specified in the "outnamelist" (if Post-processing is disabled) in the application, so that i can process them.

Please quote the portion of the application where I can get the model output before it Post-process it for labels.

Thanks and Regards,

Vyom Mishra

Please find my answers inline :

I have used this parameter " outDataNamesList" for importing the model and provided all three names of the layer to the list.

 "in this case, TIDL will return these layers as output"

By the above statement you mean to say that all three layers output will be stored in the "output.bin" that is the model output stored in the binary file.

Anshu >>>output.bin is a test-bench code which writes the final output of each layer in memory. For your post processing you should directly use the buffers coming out of TIDL.The properties of these buffers can be found from sTIDL_IOBufDesc_t structure which is output of TIDLimport tool. Refer the below document for understanding it :

http://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/latest/exports/docs/tidl_j7_01_01_00_10/ti_dl/docs/user_guide_html/md_tidl_fsg_io_tensors_format.html

1. If it is the case, then will it be writing the output of the layers(i.e., float values) in the contagious form to the binary file like (layer109,layer120,layer127).bin.      Kindly correct me with the understanding of the output dumps of the model.

Anshu >>> Outputs are written as per the output datatypes. If output of a layer is fixed point output then it will be coming as fixed point with a corresponding scale associated with it which will come as part of TIDL_outArgs. If you want a floating point output, you will have to divide the fixed point output by corresponding scale.

2. Please quote the portion of the code in the application where I can get this tidl model output

Anshu >>> Please refer tidl_WriteNetOutputMem function in tidl_tb.c file.

As per my understanding, in the App

app_create_graph_draw_detections(obj->graph, &obj->drawDetectionsObj, obj->tidlObj.output_tensor_arr[0], obj->scalerObj.output2.arr[0]);

Here  "obj->tidlObj.output_tensor_arr[0] " has the labels for the frame(if post-processing is enabled). Is it possible to access the tidl model output with the feature dumps specified in the "outnamelist" (if Post-processing is disabled) in the application, so that i can process them.

Anshu >>> Yes. I will ask Shyam to let you know where exactly to tap these outputs in vision Apps.

Please quote the portion of the application where I can get the model output before it Post-process it for labels.

Anshu >>> For post processing you can refer  run_postProc function in tidl_tb_utils.c

Regards,

Anshu

I have used this parameter " outDataNamesList" for importing the model and provided all three names of the layer to the list.

 "in this case, TIDL will return these layers as output"

By the above statement you mean to say that all three layers output will be stored in the "output.bin" that is the model output stored in the binary file.

1. If it is the case, then will it be writing the output of the layers(i.e., float values) in the contagious form to the binary file like (layer109,layer120,layer127).bin.      Kindly correct me with the understanding of the output dumps of the model.

2. Please quote the portion of the code in the application where I can get this tidl model output

As per my understanding, in the App

app_create_graph_draw_detections(obj->graph, &obj->drawDetectionsObj, obj->tidlObj.output_tensor_arr[0], obj->scalerObj.output2.arr[0]);

Here  "obj->tidlObj.output_tensor_arr[0] " has the labels for the frame(if post-processing is enabled). Is it possible to access the tidl model output with the feature dumps specified in the "outnamelist" (if Post-processing is disabled) in the application, so that i can process them.

Please quote the portion of the application where I can get the model output before it Post-process it for labels.

Dear Sir,

Thanks for the detailed reply.

I feel I made you confuse between the testbench and "app_tidl_od".

Sir, I am seeking support in the "app_tidl_od" only.

For your information:

1) Post-processing is not part of the model

2) The model has three outputs from layer X, layer Y, layer Z

3) Separate Post-processing will be used as a kernel for processing the output(Three layers output) of the model for predictions.

Requirement:

1) We need the layer-X,layer-Y, layer-Z  output in floating format to process in the Post-processing script

Help-seeking from your side in the "app_tidl_od":

1) Is it possible to obtain the float format output of the layers in vision apps which we have mentioned in the "outnamelist" while importing?

2) I need to know the portion of the code where I can tap to access these layers output(float format) in vision apps?

kindly do the needful.

Note: We have tested the testbench output with the post-processing script using the trace dumps, now we are looking to integrate the Post-processing script with "app_tidl_od" only. Seeking support in the app. 

Thanks and Regards,

Vyom Mishra

Hi Vyom,
     I will ping Shyam to reply some of your questions. But want to add a comment on the following point :

1) Is it possible to obtain the float format output of the layers in vision apps which we have mentioned in the "outnamelist" while importing?

Anshu >>> TIDL will not convert a fixed point output to floating point output. It will provide a scale associated with fixed point output for every output. If your post processing is in floating point then you should convert these outputs to float output as part of post processing. Using trace dumps to get a floating point output of individual layer is just a debug feature and not supposed to be used for getting individual layers output in your final application.

Regards,

Anshu

Dear Sir,

Thanks for the reply.

Yeah, we used to do the same when we were working on TDA2x. to convert the fixed point to floating point output for layer-by-layer matching.

It would be kind of you if Shyam Sir can reply as soon as possible.

Gentle Reminder!

Thanks and Regards,

Vyom Mishra

Dear Sir,

Thanks for the Reply.

"If TIDL is providing you with these intermediate outputs then I'd like to know the number of output tensors that the imported model is reporting"

Vyom>> We have the model which has three outputs, specifically mentioned in the import config file in "outnamelist" parameter

"The app_tidl_od application will create as many output tensors as indicated by the imported binary file."

Vyom>> This I have cross-checked and found it to be same as the number of output we are expecting from model i.e., 3

"Also can you please provide some details on your usecase? How you intend to consume this information and how you plan to run this on EVM"

three outputs extracted from TIDL will be parsed to Post-Processing Script[separately written in C code:: this code will be running on DSP]

I have some more queries regarding the same :

1) In a model where post-processing has enabled, the output of the model is stored in the "&output_tensors[i]"

Query: Does this output tensor holds the layer feature dumps or post-processed output of the model if Post-processing is enabled in the model?

2) In a model where post-processing has disabled, the output of the model is stored in the "&output_tensors[i]"

Query: Does this output tensor holds the layer feature dumps if Post-processing is disabled?

Kindly do the needful.

Thanks and Regards,

Vyom Mishra

Vyom,

Can we close this thread as we had an offline discussion?

Regards,
Shyam

Dear Sir,

we have several observations for which we need to continue this thread.

As discussed in the offline call.

We extracted the TIDL output directly from the tensor and provided the same to the custom Post-Processing kernel. We observed that number of detected boxes were 50, So we planned an exercise to know if the correct output from the TIDL  is fed to the Post-Processing.

So, we cross-checked the TIDL Output tensor and the traces generated from the model for the particular output layers of the model, which matches.

One more exercise was done with the StandAlone Post-processing script which is below:

As you know that in our model "PostProcType" is disabled and we are using our custom Post-processing in the app_tidl_od.

With the same input image( difference in the format as app_tidl_od expect .yuv format and .bmp was supplied to the PC-Emulation(StandAlone))

When we tried to process the trace dumps generated from app_tidl_od through our StandAlone Post-processing script, there were no detections observed whereas the same imported model generated the traces from PC-Emulation(StandAlone) detects correctly.

When I tried to match the trace dumps(float.bin) of app_tidl_od and PC-Emulation(StandAlone) it didn't match.

Query:

1) Why this mismatching is observed between the traces of PC-Emulation(StandAlone) and app_tidl_od?

2)  Does StandAlone PC-Emulation and app_tidl_od have the traces format difference(i.e., NCHW/NHWC)?

     (As per our understanding TIDL output on StandAlone PC-Emulation/Target is in NCHW format)

Kindly do the needful.

Thanks and Regards,

Vyom Mishra

Hi Vyom,

This thread seem to be open for a while, was there any progress on the above issues? Do you need any help with some of the queries? I highly encourage you to open multiple threads as required instead of keeping one thread alive on various issues.

Regards,
Shyam