This thread has been locked.
If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.
TIDL automatically quantize the network. If I want to specify the quantizaiton method for each layer, is there any example?
Please specify what processor you are using, and which software SDK, resp. version.
I use TDA4VM and want to do inference using C71. Is there any way to import my own quantization configuration, e.g., import the quantization params using prototxt?
I find caffe-jacinto has quantization params. If I trained model in caffe-jacinto, and the trained prototxt has QuantizationParameter for each layer. Will the TIDL import the QuantizationParameter into the .bin file automatically?
message QuantizationParameter {
optional QuantizationParameter_Precision precision = 1 [default = QuantizationParameter_Precision_DYNAMIC_FIXED_POINT];
optional QuantizationParameter_Rounding rounding_scheme = 2 [default = QuantizationParameter_Rounding_NEAREST];
optional bool power2_scale_weights = 3 [default = false];
optional bool power2_scale_activations = 4 [default = false];
// Dynamic fixed point params
message QParams {
optional bool quantize = 1 [default = false];
optional uint32 bitwidth = 2 [default = 8];
optional int32 fracbits = 3 [default = 0];
optional bool unsigned_data = 4 [ default = false];
optional bool unsigned_quant = 5 [ default = false];
optional float scale_target = 6 [default = 1.0];
optional float scale_applied = 7 [default = 1.0];
optional int32 shiftbits = 8 [default = 0];
optional float offset = 9 [default = 0];
optional float min = 10 [default = 0];
optional float max = 11 [default = 0];
}
repeated QParams qparam_in = 5;
repeated QParams qparam_w = 6;
repeated QParams qparam_out = 7;
optional int32 quantized_infer_count = 8 [ default = 0];
}
/* * * Copyright (c) {2015 - 2017} Texas Instruments Incorporated * * All rights reserved not granted herein. * * Limited License. * * Texas Instruments Incorporated grants a world-wide, royalty-free, non-exclusive * license under copyrights and patents it now or hereafter owns or controls to make, * have made, use, import, offer to sell and sell ("Utilize") this software subject to the * terms herein. With respect to the foregoing patent license, such license is granted * solely to the extent that any such patent is necessary to Utilize the software alone. * The patent license shall not apply to any combinations which include this software, * other than combinations with devices manufactured by or for TI ("TI Devices"). * No hardware patent is licensed hereunder. * * Redistributions must preserve existing copyright notices and reproduce this license * (including the above copyright notice and the disclaimer and (if applicable) source * code license limitations below) in the documentation and/or other materials provided * with the distribution * * Redistribution and use in binary form, without modification, are permitted provided * that the following conditions are met: * * * No reverse engineering, decompilation, or disassembly of this software is * permitted with respect to any software provided in binary form. * * * any redistribution and use are licensed by TI for use only with TI Devices. * * * Nothing shall obligate TI to provide you with source code for the software * licensed and provided to you in object code. * * If software source code is provided to you, modification and redistribution of the * source code are permitted provided that the following conditions are met: * * * any redistribution and use of the source code, including any resulting derivative * works, are licensed by TI for use only with TI Devices. * * * any redistribution and use of any object code compiled from the source code * and any resulting derivative works, are licensed by TI for use only with TI Devices. * * Neither the name of Texas Instruments Incorporated nor the names of its suppliers * * may be used to endorse or promote products derived from this software without * specific prior written permission. * * DISCLAIMER. * * THIS SOFTWARE IS PROVIDED BY TI AND TI'S LICENSORS "AS IS" AND ANY EXPRESS * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL TI AND TI'S LICENSORS BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED * OF THE POSSIBILITY OF SUCH DAMAGE. * */ #include <stdint.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include "itidl_ti.h" int32_t main(int32_t argc, char *argv[]) { if(argc < 5) { printf("Usage %s <tidl_model_file> <layer_number> <float_min> <float_max>\n" ,argv[0]); exit(0); } FILE * fptr; int32_t netSize; fptr = fopen((const char *)argv[1], "rb"); if (!fptr) { printf("Could Not Open File %s\n", argv[1]); exit(0); } fseek(fptr, 0L, SEEK_END); netSize = ftell(fptr); fseek(fptr, 0L, SEEK_SET); sTIDL_Network_t *net = (sTIDL_Network_t *)malloc(netSize); if(net == NULL) { printf("Could NOT Allocate memory for net of size %d\n", netSize); exit(0); } int32_t freadStatus = fread(net,1, netSize, fptr); if ( freadStatus != netSize ) { printf("Could not read %d bytes from file %s\n", (netSize - freadStatus), argv[1]); exit(0); } fclose(fptr); sTIDL_Layer_t *TIDLLayer = &net->TIDLLayers[atoi(argv[2])]; printf("Current Min and Max for Layer %d : %f , %f\n", atoi(argv[2]), TIDLLayer->outData[0].minTensorValue,TIDLLayer->outData[0].maxTensorValue); TIDLLayer->outData[0].minTensorValue = atof(argv[3]); TIDLLayer->outData[0].maxTensorValue = atof(argv[4]); printf("Updated Min and Max for Layer %d : %f , %f\n", atoi(argv[2]), TIDLLayer->outData[0].minTensorValue,TIDLLayer->outData[0].maxTensorValue); fptr = fopen((const char *)argv[1], "wb+"); if (!fptr) { printf("Could Not Open File %s\n", argv[1]); exit(0); } int32_t fwriteStatus = fwrite(net,1, netSize, fptr); if ( fwriteStatus != netSize ) { printf("Could not write %d bytes to file %s\n", (netSize - fwriteStatus), argv[1]); exit(0); } fclose(fptr); return (0); }
If you are referring to the feature vector range. After importing the model, you can use the attached code to update the feature vector range.
For Parameter quantization, you can update Quantization logic in the import tool are use QAT.
Refer below documentation for more details