yukkyo/voc2coco

Convert VOC format XMLs to COCO format json

voc2coco's Issues

Generated image IDs are non-unique

Firstly, thanks for creating this script, it was a great help to me.

When I first ran it, it worked almost perfectly, but with one problem - the COCO format image IDs were all over the place, many non-unique (many 0s for example) which breaks the COCO format. I saw how you're generating them as a function of the filename, and given the image IDs have no VOC equivalent, I think it would make more sense to do a strict ordering per image.

I did a hacky solution for now, I'm leaving this issue open so I can come back to it later and open a PR with a fix. If anybody else is having this problem, look for img_id and you can try incrementing it manually for the moment.

Annotations and area calculation.

@yukkyo Good attempt, however, the problem with annotations written in the .json have specific problem for all annotations.

1. XMIN and YMIN are always reduced by 1 pixel.
2. XMAX and YMAX are assigned values that cannot be traced to original values.
3. Above issues also cause wrong calculations of area of bboxes.

Syntax error

I get the following error whenever I run this script, even with the sample data provided here.

  File "voc2coco.py", line 10
    def get_label2id(labels_path: str) -> Dict[str, int]:
                                ^
SyntaxError: invalid syntax

Any help would be greatly appreciated.

UnicodeDecoderError in win10

> E:\DOTA_dataset\coco2017>python voc2coco.py  --ann_dir annotations/train_xml --ann_ids train.txt --labels labels.txt --output train2017.json --ext xml
> Traceback (most recent call last):
>   File "voc2coco.py", line 151, in <module>
>     main()
>   File "voc2coco.py", line 140, in main
>     annpaths_list_path=args.ann_paths_list
>   File "voc2coco.py", line 32, in get_annpaths
>     ann_ids = f.read().split()
> UnicodeDecodeError: 'gbk' codec can't decode byte 0xff in position 0: illegal multibyte sequence

can anyone help me?

getting an error in labels.txt

Hi,
Hope you are doing great.
I am tring to transfer .xml files to coco format, my labels is
Not a target vesse and Target Vessel
I get this error, your kind support plz

Start converting !
0%| | 0/350 [00:00<?, ?it/s]
Traceback (most recent call last):
File "voc2coco.py", line 152, in
main()
File "voc2coco.py", line 143, in main
convert_xmls_to_cocojson(
File "voc2coco.py", line 106, in convert_xmls_to_cocojson
ann = get_coco_annotation_from_obj(obj=obj, label2id=label2id)
File "voc2coco.py", line 62, in get_coco_annotation_from_obj
assert label in label2id, f"Error: {label} is not in label2id !"
AssertionError: Error: Not a target vessel is not in label2id !

How to make sample/text.txt ？

How to make sample/text.txt ？

I would like to see the output files

I see that you include the output files in your .gitignore file but I think it would be helpful to include them in your repo so that people (like me) can see an example of the output without having to run the code. Thank you!

License and sharing ? =)

Hello,

great code yukkyo, thank you ! I would like to use your script in a pipeline that will be released open-source (BSD), and for user convenience, I'd like to provide directly (with all credits to you of course). But I did not find any license disclaimer on your repo. Would you add one ? Otherwise, it is no problem, I will simply instruct users to grab your script here :)

Best,
Clem

Problems with white spaces in ann_paths_list

The script is not working when using pathes that contain whitespaces in the ann_paths_list.

A simple solution would be to split by "\n" in the get_annpaths function.

why '-1' in state which convert xmin and ymin

why '-1' in state which convert xmin and ymin

In the class get_coco_annotation_from_obj

Made little adjustment to convert easier

Thank you very much for the code you provided. I made a small modification and it might be easier to use.

In practical applications, annotations are often not accurate, and sometimes need to be cleaned, or data needs to be moved. Therefore, it is often inaccurate to directly read the path in xml, and the location of pictures and annotations has often changed. I think that in practical applications, like xml annotations, if the image name is the same as the xml name, it can be considered as a group, which is better.

Specific method: read the name of the corresponding image through the xml path and write it into the json file. (function get_image_info and function convert_xmls_to_cocojson)

def get_image_info(annotation_root, imgname, extract_num_from_imgid=True):
filename = imgname # First Change
img_name = os.path.basename(filename)
img_id = os.path.splitext(img_name)[0]
if extract_num_from_imgid and isinstance(img_id, str):
img_id = int(re.findall(r'\d+', img_id)[0])
size = annotation_root.find('size')
width = int(size.findtext('width'))
height = int(size.findtext('height'))
image_info = {
'file_name': filename,
'height': height,
'width': width,
'id': img_id
}
return image_info

def convert_xmls_to_cocojson(annotation_paths: List[str],
label2id: Dict[str, int],
output_jsonpath: str,
extract_num_from_imgid: bool = True):
output_json_dict = {
"images": [],
"type": "instances",
"annotations": [],
"categories": []
}
bnd_id = 1 # START_BOUNDING_BOX_ID, TODO input as args ?
print('Start converting !')
for a_path in tqdm(annotation_paths):
# Read annotation xml
ann_tree = ET.parse(a_path)
ann_root = ann_tree.getroot()
imgn = (a_path.split('/')[-1]).split('.')[0] + '.jpg' # Second Change: get the image name from xml path
img_info = get_image_info(annotation_root=ann_root,
extract_num_from_imgid=extract_num_from_imgid,
imgname=imgn)
img_id = img_info['id']
output_json_dict['images'].append(img_info)
for obj in ann_root.findall('object'):
ann = get_coco_annotation_from_obj(obj=obj, label2id=label2id)
ann.update({'image_id': img_id, 'id': bnd_id})
output_json_dict['annotations'].append(ann)
bnd_id = bnd_id + 1
for label, label_id in label2id.items():
category_info = {'supercategory': 'none', 'id': label_id, 'name': label}
output_json_dict['categories'].append(category_info)
with open(output_jsonpath, 'w') as f:
output_json = json.dumps(output_json_dict)
f.write(output_json)

Using generated .json with CocoDetection dataset class from PyTorch

Hello everyone,

First of all, thanks for sharing, this repo has been so useful for me. I will explain a possible change to the script for those who want to use the CocoDetection class from PyTorch to load the dataset, just in case anyone faces the same issues as me.

I'm using PyTorch to train a Faster RCNN object detector, and all the model expects the data in the form defined by CocoDetection dataset class from PyTorch. The .json created with the script has one issue when using it with the mentioned dataset class. It has to do with image ids. The problem is that image ids are created as strings inside the .json, what then causes problems in CocoDetection class when loading images, as it uses the image id to lod the image and checks internally several times if the id is an integer, following a different loading path if not and causing exceptions.

In order to solve this, the easiest solution I've found is to just define the image ids as integers in the script, rather than strings. As images are labeled year_xxxxxxx.jpg, the id will be year_xxxxxxx. If this string is converted to integer, when converting the ids to a integers it just will convert them to yearxxxxxxx, therefore preserving the uniqueness of the id and not causing error when loading images with CocoDetection class.

The solution is to modify the voc2coco.py in the lines 51 to 56 of the script:
Current version:

image_info = {
        'file_name': filename,
        'height': height,
        'width': width,
        'id': img_id
    }

Modification:

image_info = {
        'file_name': filename,
        'height': height,
        'width': width,
        'id': int(img_id)
    }

And that's it, just casting the img_id variable to integer.

I hope this will be useful for someone!