PROCESSOR-SDK-J721E: Sevaral artifacts

Hi 

It seems that you have two subgraphs as the model may have layers that are not supported in TIDLRT and has to be run on ARM. 

If you look at the file runtimes_visualization...svg (open in chrome or firefox) it shows the subgraphs created.

Does this answer your first question? If you share the .svg file, then we can take a look that as well.

Hello,

Thank you for your reply.
I believe that your answer include what I want to know.

Attached is my runtime_visualization.svg. Actually, this file has two kind of highlighted parts.(One is highlighted blue. And another is highlighted green.)
However, since I'm not sure that this file has expected result which you described, so could you please confirm attached file ?

BR,runtimes_visualization.zip

How did you train that model (which repository did you use)? I can see QUANTIZE layer in between and at the end which is causing multiple subgraphs to be created. 

But this is not the reason for:

ValueError: Could not open '/opt/model_zoo/imagecls-Custom_tflitert_xxxxx. 

It may be a path problem or something else.

>How did you train that model (which repository did you use)?
I did not use TI tool(modelmaker) but I used another tool.

>It may be a path problem or something else.
Umm, I will review overall.

BR,

Hello,

When I added artifacts_folder and model_folder as environment variable, I could avoid first error.
However, I got different error.
* ValueError: Didn't find op for builtin opcode 'CAST' version '2' registration failed

I think that it seems that kind of this error may be related to tflite runtime version...
But do you have any idea for this error ?

BR,

May be that operator is not supported in TIDL. 

Hi,
 
I checked another model.
Here is svg file of "TFL-CL-0000-mobileNetV1-mlperf".
It seems that "CAST" operator is used for tensorflow lite model.
So, I think that there is another reason...
What do you think ?

BR,

Hmm.. May be you are right. I can forward this thread to a colleague and let him take a look.

Hi,

The cast, add and mul operators you see in above tflite model are added as part of edgeai-benchmark scripts in order to absorb the pre-processing (mean and scale) into the model, and are converted to a different layer (data convert) internally which gets supported on TIDL-RT. 

Standalone Cast layer as part of tflite model is not supported on TIDL-RT. I also suspect this has something to do with version, but I haven't specifically seen this issue before. Which tflite version are you using?

Regards,

Anand

Thanks Anand for that insight. 

Ryuuichi, can you try running edgeai-benchark based compilation with input_optimization set to False

github.com/.../settings_base.yaml

Hello

Thank you for your reply.
I could avoid "CAST" error by changing input_optimization parameter.
Then, I have another error.
* if 'metric' in params:
TypeError: argument of type 'NoneType' is not iterable

I think that the reason may "run_dir" parameter for param.yaml.
According to param.yaml file of "TFL-CL-0000-mobileNetV1-mlperf", "run_dir" is defined in "metric".
I'm checking yaml file and benchmark_custom.py file but it seems that there is no definition code.
Do you know which file define above parameter ?

BR,

run_dir will be created internally. see here: https://github.com/TexasInstruments/edgeai-benchmark/blob/master/edgeai_benchmark/sessions/basert_session.py#L344

1. Can you pull the latest from edgeai-benchmark

2. Run the script run_custom_pc.sh 

Are the classification models in this script working for you?

Now try your model - If you are still getting issue, we may need to have a call to resolve this.

Note 1: I was able to run the script run_custom_pc.sh and it worked.

Note 2: You can reduce these three parameters for quick experimentation: https://github.com/TexasInstruments/edgeai-benchmark/blob/master/settings_base.yaml#L36

# number of frames for inference
num_frames : 10 #10000 #50000

# number of frames to be used for post training quantization / calibration
calibration_frames : 10 #50 #100

# number of itrations to be used for post training quantization / calibration
calibration_iterations : 10 #50 #100

Hello,

run_dir will be created internally. see here: https://github.com/TexasInstruments/edgeai-benchmark/blob/master/edgeai_benchmark/sessions/basert_session.py#L344

It seems that this code for parameter of "session".
Actually, "session" also have same parameter, however "run_dir" of "session" can be set correcly.
The problem is for only "run_dir" of "metric".

If L344 apply to both "run_dir" of "session" and "metric", I can not understand why setting parameter is different.
However, do you believe that above code can be applied to both "session" and "metric" ?

>Are the classification models in this script working for you?
I'm sorry that I could not understand meaning of above.
What do you mean ?

BR,

Reading your question again, I understood it better. Some of the accuracy metric calculations require temporary files to be created. For example the detection metric calculations in mmdetection. This is why run_dir is passed to the metric calculation. It is set here:

https://github.com/TexasInstruments/edgeai-benchmark/blob/master/edgeai_benchmark/pipelines/accuracy_pipeline.py#L281

Hello,

At first, as you said, I performed git pull for edgeai_benchmark and changed setting_base.yaml.
However, same error occur.

I am checking L281. when I get question, I will let you know again.

BR,

Glad that you are able to run the model on EVM.