- Ask a related questionWhat is a related question?A related question is a question created from another question. When the related question is created, it will be automatically linked to the original question.
Hello, TI experts：
We have three questions about using TIDL to quantize our stereo-CNN model.
The environment we used:
cuda 11.0
pytorch 1.11.0
torchvision_tidl ：github.com/.../edgeai-torchvision (latest version)
python : 3.8.12
The model we used is named FADNet, and the related code about model structure and training flow have be attached in the attachment. To decrease the attachment size, we only put one pair of stereo pictures in the fold.
For quantizing the FADNet, we have three questions:
1. The FADNet is a stereo depth-estimation CNN,and it used a weight-sharing twin structure in the feature extraction part. Our question is, how to share the weights in the twin structure, is it right we used as below:
Line 236 in the model.py ，conv1,conv2,conv3 are the share-weight layers
# weight share Is this correct?
self.conv1_2 = self.conv1
self.conv2_2 = self.conv2
self.conv3_2 = self.conv3
2. There is a cost volume (the function named build_corr) operation in the FADNet, but this operation is not supported in the TIDL now, how should we handle this operation?
Line 113 in the model.py
def build_corr(img_left, img_right, max_disp=40, zero_volume=None):
B, C, H, W = img_left.cpu().detach().numpy().shape
if zero_volume is not None:
tmp_zero_volume = zero_volume # * 0.0
# print('tmp_zero_volume: ', mean)
volume = tmp_zero_volume
else:
volume = img_left.new_zeros([B, max_disp, H, W])
for i in range(max_disp):
if (i > 0) & (i < W):
volume[:, i, :, i:] = (img_left[:, :, :, i:] * img_right[:, :, :, :W - i]).mean(dim=1)
else:
volume[:, i, :, :] = (img_left[:, :, :, :] * img_right[:, :, :, :]).mean(dim=1)
volume = volume.contiguous()
return volume
3. If the network has to be truncated into two parts before and after the cost volume because the cost volume part cannot be converted, How can the network structure after the cost volume support multiple inputs and the number of input channels is greater than 4?
To solve the above questions, we have referred to the TIDL documents linked below. But we still can't solve out problems. Could you guys give some suggetions?
Hoping for your replyments.
Thank you very much!
