jsbroks/coco-annotator

:pencil2: Web-based image segmentation tool for object detection, localization, and keypoints

Features • Wiki • Getting Started • Issues • License

---

COCO Annotator is a web-based image annotation tool designed for versatility and efficiently label images to create training data for image localization and object detection. It provides many distinct features including the ability to label an image segment (or part of a segment), track object instances, labeling objects with disconnected visible parts, efficiently storing and export annotations in the well-known COCO format. The annotation process is delivered through an intuitive and customizable interface and provides many tools for creating accurate datasets.

Join our growing discord community of ML practitioner

Checkout the video for a basic guide on installing and using COCO Annotator.

Note: This video is from v0.1.0 and many new features have been added.

If you enjoy my work please consider supporting me

Features

Several annotation tools are currently available, with most applications as a desktop installation. Once installed, users can manually define regions in an image and creating a textual description. Generally, objects can be marked by a bounding box, either directly, through a masking tool, or by marking points to define the containing area. COCO Annotator allows users to annotate images using free-form curves or polygons and provides many additional features were other annotations tool fall short.

• Directly export to COCO format
• Segmentation of objects
• Ability to add key points
• Useful API endpoints to analyze data
• Import datasets already annotated in COCO format
• Annotate disconnect objects as a single instance
• Labeling image segments with any number of labels simultaneously
• Allow custom metadata for each instance or object
• Advanced selection tools such as, DEXTR, MaskRCNN and Magic Wand
• Annotate images with semi-trained models
• Generate datasets using google images
• User authentication system

For examples and more information check out the wiki.

Demo

Login Information
Username: admin
Password: password

https://annotator.justinbrooks.ca/

Backers

If you enjoy the development of coco-annotator or are looking for an enterprise annotation tool, consider checking out DataTorch.

https://datatorch.io · [email protected] · Next generation of coco-annotator

Built With

Thanks to all these wonderful libaries/frameworks:

Backend

• Flask - Python web microframework
• MongoDB - Cross-platform document-oriented database
• MongoEngine - Python object data mapper for MongoDB

Frontend

• Vue - JavaScript framework for building user interfaces
• Axios - Promise based HTTP client
• PaperJS - HTML canvas vector graphics library
• Bootstrap - Frontend component library

License

MIT

Citation

  @MISC{cocoannotator,
    author = {Justin Brooks},
    title = {{COCO Annotator}},
    howpublished = "\url{https://github.com/jsbroks/coco-annotator/}",
    year = {2019},
  }

coco-annotator's Issues

Null metadata value on load

Throws errors and won't load data if the value to a key in metadata is null.

Generate dataset only works once

Once the generate dataset is called once, the file watcher does not load any more images from the folder.

Image endpoint, height and width restrictions

Take height and width parameters of the image for it to be resized.

Export in Pascal VOC format

Support for exporting in Pascal VOC format (example)

It looks like a good tool, how to run it?

Win 10 home + docker tool box

I have recently use win 10 home and docker tool box to install the coco-annotator.
In development mode, it shows 94% after seal ERROR Failed to compile with 1 errors17:18:57.
and also in production mode, it shows
annotator_flask | Error: 'app/gunicorn_config.py' doesn't exist

How can I do with this problem? Since the app/gunicorn_config.py is already in my folder and its name without extension.

Wrong path recorded for images

In the lastest revision, the path property uses an old directory (/data) that doesn't exist anymore. E.g.:

{
  "id":1898,
  "path": "/data/datasets/obj1_day6_nov22/rgb105.jpg",
  "dataset_id": 19,
  "width": 1920,
  "height": 1080,
  "file_name": "rgb105.jpg",
  "annotated": true,
  "metadata": {}
}

How to import image to a newly created dataset?

Thanks for your open project coco-annotator in github.

I have one question here.
(OS: Linux version 4.15.0-29-generic (buildd@lgw01-amd64-057) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 )

How to import image to a newly created dataset?

What I did is followed:
1.put several images under directory "./data/datasets/test"
2.create one dataset named "test"
But it seems not working. Always showing "No images in dataset".

Need your help here.

Sometimes gets stuck "saving" during annotation

Not sure why this happens, but occasionally while annotating an image, the tool gets stuck on "saving data ...", which may prevent it from saving the most recent annotations

Add categories to existing datasets

Windows docker-compose gets unexpected mongo exit code 14

When running docker-compose on windows mongo fails:

annotator_mongodb exited with code 14

Dataset analytics page

• Average annotation size per category (bar graph)
• Average annotations per image
• Image size (max, min, average)
• Number of empty annotations

Overlapping annotation subtraction

If new annotation shares pixels with another annotation of the same category, the new annotation should be subtracted from the other (new takes priority).

Duplicate keys in mongodb when dataset is deleted

Datasets are not properly getting deleted from the mongodb instance when deleted from the main tool. Steps to reproduce:

• Create a dataset temp
• Delete a dataset temp
• Create a dataset temp

mongoengine.errors.NotUniqueError: Tried to save duplicate unique keys (E11000 duplicate key error collection: flask.dataset_model index: name_1 dup key: { : "temp"})

edit: add dataset name "temp"

"Delete Dataset" should have a confirmation box

Holy cow am I happy there's an "undo" feature.

There should definitely be an "Are you sure?" button before even the soft delete

Dataset card showing improper amount of annotated images

Display objects metadata on hover

When hovering over an annotation on the image, display any relevant information (metadata, date created, created by a user, etc.)

Importing of COCO formatted annotations

• Image annotations
• Dataset annotations

Quick Select/Magic Wand tool

Quick Selection tool to quickly “paint” a selection using an adjustable round brush tip. As you drag, the selection expands outward and automatically finds and follows defined edges in the image.

• magic-wand-js

Viewer filtering options

Add filtering options to each viewer to find models by features (name, folder, data, etc..)

• Dataset viewer
• Image viewer
• Undo list

Create, delete, re-create categories

after deleting a category, it's not possible to add it back again. It looks like the deleted category's database "delete" attribute is set to "true", but the create logic treats it as already existing.

add categories to image

To accommodate object recognition (in contrast to segmentation or detection), images need to have the option to have categories without annotations. This can be represented as a list of category ids.

image {
"id": int,
"width": int,
"height": int,
"file_name": str,
"license": int,
"flickr_url": str,
"coco_url": str,
"date_captured": datetime,
**"category_ids": [int]**
}

Propagate annotations to next/previous

Feature Request: It would be helpful if there were a capability to propagate all of the annotations on the current image to the next or previous image.

When annotating images that come from frames of a video sequence, often many of the annotations remain static from one frame to the next, with only a few annotations changing for objects that have moved.

Perhaps there could be some additional buttons next to the arrows in the upper right of the Annotator view which would trigger this functionality.

Latest build does not show images in annotator

A fresh install of the annotator no longer seems to show any images in the annotator area (e.g. http://localhost:5000/annotate/691)

I can create a new dataset and images appear in the main dataset directory, but are not visible once it gets clicked to begin annotating

Polygon line color should have maximum contrast with the background

Enhancement: It would be nice if the stroke color defaulted to whatever color shows up with maximum contrast to the image.

Next/Previous Image

Add image navigation tools inside of the editor to go to the next or previous image in the dataset.

Toast notifications

Notifications for mostly sync operations (Async are generally handled through the navbar).

Allow ad hoc categories to be created when creating dataset

Enhancement: it would be useful if coco-annotator allowed automatic creation of a list of categories at dataset creation time, such that any categories typed into the free-form list would be created if they do not already exist.

Authentication system

Tasks

• Create an authentication page
  • Register Form
  • Login Fprm
  • Form validation
• Redirect unauthenticated traffic to login page
• Require login for api endpoints
• Endpoint access (based on role)
  • datasets
  • images
  • categories
  • annotations
  • annotator
• User endpoints
  • Delete
  • View
  • Edit Info
  • Login
  • Logout
  • Register
• Navbar user section
• User settings page
• Save annotator settings to user object
• Set user perferences to the annotatior
• Admin control panel
  • Create Users
  • Delete Users
  • View Users
  • Edit Users
  • Pagnation
• vuex user module
• Dataset share option
• First user create set as admin
• Rewrite api test cases to support users

Roles

Admin

• Can edit, delete and view any dataset

Owner

• Can delete, and share datasets user created

User

• Can create, edit and view datasets shared with them

Default annotations (eye) to visible/enabled

Enhancement: it would be helpful if all of the annotations on an image are visible by default when entering an image.

Additionally, perhaps show/hide all buttons might be useful.

Simplify Polygon

If not enough points are adding while using the polygon tool, the resulting shape may not fit the desired one. In order to fix this, a new algorithm would have to be developed that simplifies the path without smoothening it.

• Ramer-Douglas-Peucker Alothrium

Hide Right Panel Distortion

Hiding the right panel distorts the image by forcing the paperjs canvas to match the size of the new frame.

Saving current annotations should not reset the tool being used

When the "save" button is clicked (or ctrl+s is used), the tool currently resets from whatever it previously was to the default "pointer".

This is a bit of an inconvenience for example when you've finished annotating one object in the image, and want to save the intermediate result. If you want to continue annotating the image, you need to re-enable the wand or polygon tool

Automatically create categories, datasets on startup from file(s)/config

Feature request: It would be nice to allow the workflow of pre-defining a bunch of categories and pre-existing datasets on startup.

I my envisioned use-case, I would volume-mount an existing datasets folder, provide some sort of manifest file containing all the categories that should exist, and a list of datasets that should be automatically created/imported (or maybe just create datasets for all the top-level directories under the /datasets folder).

This would allow me to easily migrate from other existing annotation tools to coco-annotator with minimal effort.

To stretch this a bit further, it would be nice if coco-annotator could also automatically import annotations created by these other tools (VOC format, YOLO format, etc.)

Clearer instructions on getting setup

I had some slight difficulty getting started with the tool, as a few things were ambiguous. Maybe consider adding a few more detailed steps? For example, had to figure out that:

• You first have to create a dataset
• Then you have to create categories for the dataset
• Then you have to assign a category to the dataset (if you want multiple, you need to Ctrl + click or Shift + click all of the corresponding categories)
• Then you need to move images to the data/datasets/<created_dataset> folder. I found that moving images to this folder required sudo permission on linux

A simple enhancement could be to provide these steps as (for example) a right-hand panel on the main screen, or as a "reminder" bubble that pops up when creating a dataset

Save button in api/annotate/ only sometimes works

When you create a new annotation and then click "save", the indicator in the top right corner switches from "working" to "done!". When you click save, and then download the annotations, an empty instance is returned:

e.g: {"id":24,"image_id":5,"category_id":1,"segmentation":[],"area":0,"bbox":[0,0,0,0],"iscrowd":false,"width":799,"height":533,"color":"#c110d4","metadata":{}}

But if you refresh the image after saving, and then save again, it will return the correct annotation.

Erase Tool

PaperJS Sketch Pad

No images found in directory.

Trying to run coco-annotator in Windows. I create a dataset, and put images to datasets/dataset_name No result. Generate works fine.

Ability to pan while zoomed in

When zoomed in on an object while annotating, sometimes the whole object is not in view. Thus, in order to annotate the full object, we're currently required to re-center it, and then zoom in again if needed.

An option to pan (e.g. by pressing the wheel button) would be helpful to avoid this.

(Come to think of it, a config file for "keybinds" could also be really useful)

Add metadata options

Support of adding custom metadata to Image, Annotation, Category models in the settings modal.

• Support metadata for annotations
• Support metadata for images

No images found in directory (for pre-existing directories)

I've volume-mounted an existing folder from my host machine which contains multiple datasets.
When I create a dataset (with name matching one of the dataset folders), the UI shows "No images found in directory." A quick shell into the running flask container shows that the /datasets/{dataset_name} folder does indeed contain images.

It seems like perhaps the create_dataset code should scan the directory for images if the directory already exists--else, those pre-existing images will never be found. I've added PR #48 that makes this use-case work (volume mount existing datasets, create dataset for a pre-existing dataset folder).

What do you think?

Also--thanks for sharing this repo; what you have so far looks great!

Deleting an annotation in progress should remove current polygon

When you're in the middle of drawing a polygon and click "delete", the line stays visible as it waits for you to complete a polygon.

Instead, the current polygon should disappear and be reset

An "Undo last point" macro "U"

Related to #14, being able to undo the most previous operation would help speed development a lot, as you wouldn't need to redo the whole object if you make a mistake

Error handling RGBA images

RGBA images throw an error when trying to get from api.

Export annotations only include annotated image data

Custom key bindings

As mentioned in #23. Being able to set/change custom key bindings.

Thumbnail annotation visualization

Generate thumbnail image file that visualizes annotations made on an image.

No underscores allowed in folder name?

Thanks @waspinator for helping diagnose the problem.

How come underscores aren't allowed in folder names?

Upload images/zip files to dataset though webui

Auto-match annotations (feature)

Feature Request: based on idea mentioned in #52 , it would be helpful if there was an auto-match setting for annotations such that, when a new annotation is added, a background process automatically compared the annotated image patch with other images in the dataset, and if the patch was "the same"(within some tolerance), the annotation would be automatically cloned to that image.

Note that this feature is only really useful for datasets created from video sequences.