issue while using a custom coco annotated dataset

[doursand]

issue while using a custom coco annotated dataset

[doursand]

@ppwwyyxx

Actually I gave up trying to use the register_coco_instances function and instead I created a modified version of the get_ballon_dict from the tutorial which is basically computing the bounding boxes out of the masks. I also had to define the thing_dataset_id_to_contiguous_id method to my corresponding object id

Here is the new function i created in case someone would like to try. With this new function I was able to train the model and to have the predictions of the masks without any issues

def get_peanuts_dicts(img_path,json_path):
    '''
    AD Oct 2019 : loading function to calculate the bboxes from the masks (as it does not exist so far in Detectron2)
    expected format:
    
    {'file_name': 'path/to/image',
    'height' : imgheight,
    'width': imgwidth,
    'annotations':
    [{'bbox': [xmin,ymin,xmax,ymax], 'bbox_mode': <BoxMode.XYXY_ABS: 0>,
    'segmentation': [[polygon/coordinates/x/y]],
    'category_id': 0, 'iscrowd': 0}, 
    {'bbox': [xmin2,ymin2,xmax2,ymax2], 'bbox_mode': <BoxMode.XYXY_ABS: 0>,
    'segmentation': [[polygon2/coordinates/x/y]],
    'category_id': 0, 'iscrowd': 0},
    
    etc ...]
    
    }
    INPUTS:
        img_path . str, path to the images
        json_path. str, path to the json annotation file (coco format)
    
    '''
    
    with open(json_path) as f:
        imgs_anns = json.load(f)
    dataset_dicts = []
    
    for img in imgs_anns['images']:
        record = {}
        
        filename = os.path.join(img_path,img['file_name'])
        height, width = cv2.imread(filename).shape[:2]
        image_id = img['id']

        record["file_name"] = filename
        record["height"] = height
        record["width"] = width
        
        for annos in imgs_anns['annotations']:
            if annos['image_id'] == image_id:

                objs = []
                poly = annos['segmentation']
                 
                for p in poly:
                    bbox=[]
                    x, y = p[::2] , p[1::2]
                    bbox.append(np.min(x))
                    bbox.append(np.min(y))
                    bbox.append(np.max(x))
                    bbox.append(np.max(y))
                    obj = {"bbox": bbox,
                            "bbox_mode": BoxMode.XYXY_ABS,
                            "segmentation": [p],
                            "category_id": 0,
                            "iscrowd": 0}
                    objs.append(obj)

        
        record["annotations"] = objs
        dataset_dicts.append(record)
    return dataset_dicts

[ppwwyyxx]

Sounds good. The model does expect the bounding boxes to exist and by default it does not compute it from masks (because the computed one is not necessarily equal to the annotation).
If you use a custom dataloader, you can then enable it in your mapper here:

detectron2/detectron2/data/dataset_mapper.py

Line 133 in bc4cf19

instances.gt_boxes = instances.gt_masks.get_bounding_boxes()

but of course, doing it in the dataset is also a good solution

[noobyogi0010]

In OpenCV 'NoneType' errors are generated due to not being able to read the frames. Try changing the index of d to 1.

[doursand]

thanks , however i think i know what the problem is . The coco dataset i generated is only with masks and without any bboxes, as I was creating these bboxes from the mask coordinates in the getitem method of the custom dataset I was using in torchvision segmentation example. It sounds like the register_coco_instances function does expect a "complete" coco dataset including bboxes (e.g. they are not calculated automatically from the masks). So I can close this for now

[ppwwyyxx]

The visualizer does not handle empty masks very well. This will be fixed soon.

As for why your model predicts empty masks - you can first verify your data format is correct by visualizing them just like the colab tutorial did. If the data is correct but the training fails to produce good models, we do not help people design models/parameters for their datasets.