cmusatyalab/opentpod

Open Toolkit for Painless Object Detection

OpenTPOD

Create deep learning based object detectors without writing a single line of code.

OpenTPOD is an all-in-one open-source tool for nonexperts to create custom deep neural network object detectors. It is designed to lower the barrier of entry and facilitates the end-to-end authoring workflow of custom object detection using state-of-art deep learning methods.

It provides the following features via an easy-to-use web interface.

• Training data management.
• Data annotation through seamless integration with OpenCV CVAT Labeling Tool.
• One-click training/fine-tuning of object detection deep neural networks, including SSD MobileNet, Faster RCNN Inception, and Faster RCNN ResNet, using Tensorflow (with and without GPU).
• One-click model export for inference with Tensorflow Serving.
• Extensible architecture for easy addition of new deep neural network architectures.

Demo Video

Documentation

• Motivation
• User Guide
• Installation and Administration Guide
• Developer Guide
• Thorough Description and Context in PhD Thesis Scaling Wearable Cognitive Assistance (Section 6.3)

Citations

Please cite the following thesis if you find OpenTPOD helps your research.

@phdthesis{wang2020scaling,
  title={Scaling Wearable Cognitive Assistance},
  author={Wang, Junjue},
  year={2020},
  school={CMU-CS-20-107, CMU School of Computer Science}
}

Acknowledgement

This research was supported by the National Science Foundation (NSF) under grant number CNS-1518865. Additional support was provided by Intel, Vodafone, Deutsche Telekom, Verizon, Crown Castle, Seagate, VMware, MobiledgeX, InterDigital, and the Conklin Kistler family fund.

opentpod's Issues

UI for video segments: Only 1 segment per video is displayed now

CVAT automatically segment videos. Need to provide UI for labeling multiple segments.

move training into a separate queue

[] Create a separate django-rq worker process for training to avoid blocking other background operations.
[] Add an environmental variable to allow configure how many concurrent training jobs can occur. This essentially is the number of processes to launch for rq training workers.

Instead of using timeout to estimate when the video has finished extraction, use a request to convey extraction status.

Add in search bar functions for Video and Detector Page

Production Deployment

• Use nginx as web server
• gunicorn as gateway
• use django-sendfile to send file through nginx. However, the path is absolute. This can be only tested in actual deployment (container) setup
• container setup to build and run the server

add label page to add multiple labels to multiple videos at the same time

admin Panel for deployment

Currently, the admin panel "/admin" somehow is not available when deployed through containers.

Multiple Video Training

There seems to be a problem when merging labels of the same text. The same labels for different videos have different ids. The training process seems to treat them as different labels

Multi-segment support for annotation page

Add YOLO object detector

Hi!
Thank you very much for this tool, great work! Is there any reason why you didn't add YOLO object detector to the list? Any plans to add it?

make csrf play nice with django and react

Not able to finalize installation

Hello, here the errors that I am always facing. Any advice? Thanks

andre@DESKTOP-SQG6BJ5 MINGW64 ~/OpenTPOD (master)
$ docker-compose -f docker-compose.yml -f docker-compose.prod.yml up
Creating network "opentpod_default" with the default driver
Creating opentpod-db ...
Creating opentpod-redis ...
Creating opentpod-redis ... done
Creating opentpod-db ... done
Creating opentpod ...
Creating opentpod-worker ...
Creating opentpod-trainer ...
Creating opentpod-rqscheduler ...
Creating opentpod-trainer ... done
Creating opentpod ... done
Creating opentpod-nginx ...
Creating opentpod-worker ... done
Creating opentpod-rqscheduler ... done
Creating opentpod-nginx ... done
Attaching to opentpod-redis, opentpod-db, opentpod-trainer, opentpod, opentpod-worker, opentpod-rqscheduler, opentpod-nginx
opentpod | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-db | 2021-07-19 15:03:01.319 UTC [1] LOG: starting PostgreSQL 12.0 on x86_64-pc-linux-musl, compiled by gcc (Alpine 8.3.0) 8.3.0, 64-bit
opentpod-db | 2021-07-19 15:03:01.320 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
opentpod-db | 2021-07-19 15:03:01.320 UTC [1] LOG: listening on IPv6 address "::", port 5432
opentpod-db | 2021-07-19 15:03:01.342 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
opentpod-db | 2021-07-19 15:03:01.386 UTC [20] LOG: database system was shut down at 2021-07-19 14:48:32 UTC
opentpod-db | 2021-07-19 15:03:01.399 UTC [1] LOG: database system is ready to accept connections
opentpod-nginx | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
opentpod-nginx | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
opentpod-nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
opentpod-nginx | 10-listen-on-ipv6-by-default.sh: info: can not modify /etc/nginx/conf.d/default.conf (read-only file system?)
opentpod-nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
opentpod-nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
opentpod-nginx | /docker-entrypoint.sh: Configuration complete; ready for start up
opentpod-redis | 1:C 19 Jul 15:03:00.224 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
opentpod-redis | 1:C 19 Jul 15:03:00.224 # Redis version=4.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
opentpod-redis | 1:C 19 Jul 15:03:00.224 # Configuration loaded
opentpod-redis | 1:M 19 Jul 15:03:00.226 * Running mode=standalone, port=6379.
opentpod-redis | 1:M 19 Jul 15:03:00.226 # Server initialized
opentpod-redis | 1:M 19 Jul 15:03:00.226 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
opentpod-redis | 1:M 19 Jul 15:03:00.226 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
opentpod-redis | 1:M 19 Jul 15:03:00.226 * Ready to accept connections
opentpod-rqscheduler | usage: rqscheduler [-h] [-b] [-H HOST] [-p PORT] [-d DB] [-P PASSWORD]
opentpod-rqscheduler | [--verbose] [--quiet] [--url URL] [-i INTERVAL]
opentpod-rqscheduler | [--path PATH] [--pid FILE] [-j JOB_CLASS] [-q QUEUE_CLASS]
opentpod-rqscheduler | rqscheduler: error: unrecognized arguments: me!
opentpod-trainer | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-worker | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-nginx | 2021/07/19 15:03:05 [notice] 1#1: using the "epoll" event method
opentpod-nginx | 2021/07/19 15:03:05 [notice] 1#1: nginx/1.21.1
opentpod-nginx | 2021/07/19 15:03:05 [notice] 1#1: built by gcc 8.3.0 (Debian 8.3.0-6)
opentpod-nginx | 2021/07/19 15:03:05 [notice] 1#1: OS: Linux 5.4.72-microsoft-standard-WSL2
opentpod-nginx | 2021/07/19 15:03:05 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
opentpod-nginx | 2021/07/19 15:03:05 [notice] 1#1: start worker processes
opentpod-nginx | 2021/07/19 15:03:05 [notice] 1#1: start worker process 23
opentpod-nginx | 2021/07/19 15:03:05 [notice] 1#1: start worker process 24
opentpod-nginx | 2021/07/19 15:03:05 [notice] 1#1: start worker process 25
opentpod-nginx | 2021/07/19 15:03:05 [notice] 1#1: start worker process 26
opentpod-rqscheduler exited with code 2
opentpod-trainer exited with code 1
opentpod-trainer | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-trainer | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-trainer | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-worker exited with code 1
opentpod exited with code 0
opentpod-trainer exited with code 1
opentpod | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-worker | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-worker | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-worker | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-worker | standard_init_linux.go:228: exec user process caused: no such file or directory
opentpod-rqscheduler | usage: rqscheduler [-h] [-b] [-H HOST] [-p PORT] [-d DB] [-P PASSWORD]
opentpod-rqscheduler | [--verbose] [--quiet] [--url URL] [-i INTERVAL]
opentpod-rqscheduler | [--path PATH] [--pid FILE] [-j JOB_CLASS] [-q QUEUE_CLASS]
opentpod-rqscheduler | rqscheduler: error: unrecognized arguments: me!
Exception in thread Thread-22:
Traceback (most recent call last):
File "docker\api\client.py", line 268, in _raise_for_status
File "requests\models.py", line 941, in raise_for_status
requests.exceptions.HTTPError: 409 Client Error: Conflict for url: http+docker://localnpipe/v1.41/containers/572320dbff67a3f2f41f6aec90164d1029ce870340198ad1d1f3ebf30cc5b07c/attach?logs=0&stdout=1&stderr=1&stream=1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "threading.py", line 950, in _bootstrap_inner
File "threading.py", line 888, in run
File "compose\cli\log_printer.py", line 224, in watch_events
File "compose\container.py", line 202, in attach_log_stream
File "compose\container.py", line 294, in attach
File "docker\utils\decorators.py", line 19, in wrapped
File "docker\api\container.py", line 62, in attach
File "docker\api\client.py", line 405, in _read_from_socket
File "docker\api\client.py", line 318, in _get_raw_response_socket
File "docker\api\client.py", line 270, in _raise_for_status
File "docker\errors.py", line 19, in create_api_error_from_http_exception
File "requests\models.py", line 881, in json
File "requests\models.py", line 829, in content
File "requests\models.py", line 751, in generate
File "urllib3\response.py", line 575, in stream
File "urllib3\response.py", line 518, in read
File "http\client.py", line 458, in read
File "http\client.py", line 502, in readinto
File "docker\transport\npipesocket.py", line 211, in readinto
File "docker\transport\npipesocket.py", line 20, in wrapped
RuntimeError: Can not reuse socket after connection was closed.
opentpod exited with code 1
opentpod-worker exited with code 1
opentpod-rqscheduler exited with code 2
opentpod-nginx | 172.24.0.1 - - [19/Jul/2021:15:03:27 +0000] "GET / HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36" "-"
opentpod-nginx | 2021/07/19 15:03:27 [error] 23#23: *1 connect() failed (113: No route to host) while connecting to upstream, client: 172.24.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://172.24.0.5:8000/", host: "localhost:20000"
opentpod-nginx | 172.24.0.1 - - [19/Jul/2021:15:03:30 +0000] "GET /favicon.ico HTTP/1.1" 502 559 "http://localhost:20000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36" "-"
opentpod-nginx | 2021/07/19 15:03:30 [error] 23#23: *1 connect() failed (113: No route to host) while connecting to upstream, client: 172.24.0.1, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.24.0.5:8000/favicon.ico", host: "localhost:20000", referrer: "http://localhost:20000/"

Annotation Page URL incorrect

After loading CVAT's js, annotation page's URL gets changed wrongly. Unfortunately, there seems to be no easy way to change the url back in react without redirecting.

use nginx to serve static files

video page pagination support

Annotation Page Image Loading Sometimes would get stuck

ERROR: Service 'opentpod-rqscheduler' failed to build : Build failed

The command '/bin/sh -c wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.3-Linux-x86_64.sh -O ~/miniconda.sh && /bin/bash ~/miniconda.sh -b -p /opt/conda && rm ~/miniconda.sh && /opt/conda/bin/conda update -n base -c defaults conda && /opt/conda/bin/conda clean -tipsy && ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && echo ". /etc/profile" >> ~/.bashrc # used by docker exec/run' returned a non-zero code: 2

No opentpod:stable container image on DockerHub

I get "ERROR: pull access denied for opentpod, repository does not exist or may require 'docker login': denied: requested access to the resource is denied" when i run "docker-compose -f docker-compose.yml -f docker-compose.prod.yml up"
I modified the .env file but I was wondering if i did something wrong. Any idea how to solve this issue? Thanks in advance

django pagination frontend handling

Enabled django pagination when getting video infos. Need to make frontend to work with such pagination.

Logzero's content is not showing up in gunicorn logs

Current visualization does not support multiple users