rohitgeo/singleshotdetector

SingleShotDetector implementation using Fastai v3

singleshotdetector's Issues

Inference on onnx/mlmodel

Has anyone succeeded in porting this for inference on mobile? I've converted it, but cannot figure out to to transform the bounding box and classification outputs into x,y,w,h.

I thought I might be able to grab the highest confidence in the class I'm looking for and use that row index to find the corresponding bounding box, but the bounding box is not in the typical range (between 0 and 1).

ans = preds['confidence'][0].argmax(axis=0)
coords = preds['coordinates'][0][ans[3]] # ans[3] corresponds to the class I'm looking for
coords_softmax = softmax(coords, axis=0)
print(coords)
[  0.35327148   2.9023438    1.5244141  -10.9296875 ]

Below is my conversion code.

size = (224,224)

class ImageScale(nn.Module):
    def __init__(self):
        super().__init__()
        mean_r = torch.full((1, size[0], size[1]), 0.485, device=torch.device("cuda"))
        std_r = torch.full((1, size[0], size[1]), 0.229, device=torch.device("cuda"))
        mean_g = torch.full((1, size[0], size[1]), 0.456, device=torch.device("cuda"))
        std_g = torch.full((1, size[0], size[1]), 0.224, device=torch.device("cuda"))
        mean_b = torch.full((1, size[0], size[1]), 0.406, device=torch.device("cuda"))
        std_b = torch.full((1, size[0], size[1]), 0.225, device=torch.device("cuda"))
        
        self.denominator = torch.full((1, size[0], size[1]), 255., device=torch.device("cuda"))
        self.means = torch.cat((mean_r, mean_g, mean_b), 0)
        self.stds = torch.cat((std_r, std_g, std_b), 0)

    def forward(self, x): 
        normalized = torch.div(x,self.denominator)
        numerator = torch.sub(normalized,self.means)
        out = torch.div(numerator,self.stds)
        return out.unsqueeze(0)

final_model = [ImageScale()] + [ssd.learn.model]
final_model = nn.Sequential(*final_model)

model_name = "ssd_resnet_5_epochs.onnx"
dummy_input = Variable(torch.randn(3, size[0], size[1])).cuda()
torch.onnx.export(final_model, dummy_input, model_name, input_names = ['image'], output_names=['confidence','coordinates'])
onnx_model = onnx.load(model_name)

mlmodel = convert(onnx_model, image_input_names = ['image'], target_ios='13')
mlmodel.input_description['image'] = 'Image'
mlmodel.output_description['coordinates'] = 'Coordinates'
mlmodel.output_description['confidence'] = 'confidence'
mlmodel.save('ssd_resnet_5_epochs.mlmodel')

Bug in ssd.unfreeze()

Simple bug - 'self' attribute is missing in the definition of the unfreeze() function for SingleShotDetector Class.

Improvement: Also, fit_one_cycle() could be defined and used instead of simple fit() for nn training

Can't predict on new images

I've trained my model for quite a bit now on different data which works great. but I'm having trouble usiing predict() on images that are not of training, validation. or test sets. I got this error when attempting to predict:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-53-3dd4dc20a925> in <module>
----> 1 test_prediction = ssd_model.learn.predict(test_images[0])

~/anaconda3/envs/fastai/lib/python3.7/site-packages/fastai/basic_train.py in predict(self, item, **kwargs)
    363             if norm.keywords.get('do_y',False): pred = self.data.denorm(pred)
    364         ds = self.data.single_ds
--> 365         pred = ds.y.analyze_pred(pred, **kwargs)
    366         out = ds.y.reconstruct(pred, ds.x.reconstruct(x[0])) if has_arg(ds.y.reconstruct, 'x') else ds.y.reconstruct(pred)
    367         return out, pred, res[0]

~/dev/.../ssdoil.py in analyze_pred(self, pred, thresh, nms_overlap, ssd)
     60     def analyze_pred(self, pred, thresh=0.5, nms_overlap=0.1, ssd=None):
     61         # def analyze_pred(pred, anchors, grid_sizes, thresh=0.5, nms_overlap=0.1, ssd=None):
---> 62         b_clas, b_bb = pred
     63         a_ic = ssd._actn_to_bb(b_bb, ssd._anchors.cpu(), ssd._grid_sizes.cpu())
     64         conf_scores, clas_ids = b_clas[:, 1:].max(1)

ValueError: not enough values to unpack (expected 2, got 1)

The problem seems to be in the SSDObjectCategoryList analyze_preds() method