jagin/detectron2-pipeline

Modular image processing pipeline using OpenCV and Python generators powered by Detectron2.

Detectron2 pipeline

Modular image processing pipeline using OpenCV and Python generators powered by Detectron2.

For detailed description how to construct image processing pipeline using OpenCV and Python generators read the following Medium stories in order:

• Modular image processing pipeline using OpenCV and Python generators
• Video processing pipeline with OpenCV
• How to embed Detectron2 in your computer vision project
• PoseFlow - real-time pose tracking

Setup environment

This project is using Conda for project environment management.

Setup the project environment:

$ conda env create -f environment.yml
$ conda activate detectron2-pipeline

or update the environment if you git pull the repo previously:

$ conda env update -f environment.yml

Install Detectron2 in a folder one level above the detectron2-pipeline project folder

$ cd ..
$ git clone https://github.com/facebookresearch/detectron2.git
$ cd detectron2
$ git checkout 3def12bdeaacd35c6f7b3b6c0097b7bc31f31ba4
$ python setup.py build develop

We checkout 3def12bdeaacd35c6f7b3b6c0097b7bc31f31ba4 commit to ensure that you can use the code out of the box with this repo.

If you got any problems with Detectron2 installation please refer to INSTALL.md.

Demo

Run the command to execute inferences on images from the selected folder:

$ python process_images.py -i assets/images/friends -p

By default the instance segmentation model is used from configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml. You can try other models with --config-file option, for example:

$ python process_images.py -i assets/images/friends -p --config-file configs/COCO-Keypoints/keypoint_rcnn_R_50_FPN_3x.yaml

For video processing run:

$ python process_video.py -i assets/videos/walk.small.mp4 -p -d -ov walk.small.mp4

Tests

pytest is used as a test framework. All tests are stored in tests folder. Run the tests:

$ pytest

Resources and Credits

• For Unix like pipeline idea credits goes to this Gist
• The source of the example images and videos is pixbay
• Some ideas and code snippets are borrowed from pyimagesearch
• Color constants from Python Color Constants Module
• Detectron2
• Faster video file FPS with cv2.VideoCapture and OpenCV
• Applications of Foreground-Background separation with Semantic Segmentation
• Pose tracking based on PoseFlow: Efficient Online Pose Tracking

License

MIT License

detectron2-pipeline's Issues

tracking took 15 seconds on >50 detections

hey @jagin , when you tested this pipeline, have you ever experimented on the number of detections vs how long it would took to solve the hungarian matching? im testing this and it took as long as 15 seconds on best_matching_hungarian after some profiling. Do you have any pointers where i can look to improve this or is this the limitations of the algorithm?

Thanks!

can't compile detectron2 pillow issue

Hi
it looks like pillow 7 breaks compatibility with pytorch and needs to be updated or force pillow to lower version

https://discuss.pytorch.org/t/cannot-import-name-pillow-version-from-pil/66096

AssertionError: Number of gpus or cpus must be specified

Hello could I ask the problem: number of gpus or cpus must be specified.
Like the attached:

Has anyone met this? Thanks ahead!

Detectron2 external projects

Hey @jagin,

I would like to ask you how can I connect Detectron2 external projects(like this one: https://github.com/youngwanLEE/centermask2) to the pipeline?

Thanks!

[Question]Any possible way to make person reidentification project based on detectron2

Could you please share your thoughts basically on 2 topics of my interest:

• Doing person-only detection not all the objects
• After that, doing person reidentification but again using detectron2.

Any samples, inputs, maybe articles are highly appreciated as well.

The video part is not working

The already existing issue on single process is still preveleant along with that the video analysis is not working. A empty directory is made and also according to me there is a problem in frame reading.

Display only certain pred_classes

Hi,

Thank you for this great work.
I want to run this as inference on my video but to display only certain class label.
In the original detectron2 I know I can do something like this:
predictions = predictor(frame)
instances = predictions["instances"]
category_detections = instances[instances.pred_classes == 0]

and display the output of draw_instance_predictions.

Is it possible to do something like this?

Thank you.

question: is there an easy way to get only class labels out for each image?

Hi, very nice work!

Can i ask if there is an easy way to get only class labels out for each image?

For my project i don't need visualizations, instead i'd love a dataframe with class labels.

Thank you

Not an issue — Pipeline

Hey @jagin, awesome repository! what online resources do you suggest for learning about pipeline (how does it work,...)? Thanks.

cpu+gpu is slower than gpu，How to solve？thanks！

initial test, multiprocessor mode not working single process works

I am getting the following error

This may be due to my setup, the --single-process flag works fine.
Without --single-process the following error is emitted.

RuntimeError: cuda runtime error (801) : operation not supported at C:\w\1\s\tmp_conda_3.7_104508\conda\conda-bld\pytorch_1572950778684\work\torch/csrc/generic/StorageSharing.cpp:245
Traceback (most recent call last):
File "C:\test\lib\multiprocessing\queues.py", line 236, in _feed
obj = _ForkingPickler.dumps(obj)
File "C:\test\lib\multiprocessing\reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "C:\test\lib\site-packages\torch\multiprocessing\reductions.py", line 242, in reduce_tensor
event_sync_required) = storage.share_cuda()
RuntimeError: cuda runtime error (801) : operation not supported at C:\w\1\s\tmp_conda_3.7_104508\conda\conda-bld\pytorch_1572950778684\work\torch/csrc/generic/StorageSharing.cpp:245
Traceback (most recent call last):
File "C:\test\lib\multiprocessing\queues.py", line 236, in _feed
obj = _ForkingPickler.dumps(obj)
File "C:\test\lib\multiprocessing\reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "C:\test\lib\site-packages\torch\multiprocessing\reductions.py", line 242, in reduce_tensor
event_sync_required) = storage.share_cuda()
RuntimeError: cuda runtime error (801) : operation not supported at C:\w\1\s\tmp_conda_3.7_104508\conda\conda-bld\pytorch_1572950778684\work\torch/csrc/generic/StorageSharing.cpp:245
Process _PredictWorker-1:
Traceback (most recent call last):
File "C:\test\lib\multiprocessing\process.py", line 297, in _bootstrap
self.run()
File "C:\g\vn\lc\detectron2-pipeline\pipeline\libs\async_predictor.py", line 32, in run
task = self.task_queue.get()
File "C:\test\lib\multiprocessing\queues.py", line 94, in get
res = self._recv_bytes()
File "C:\test\lib\multiprocessing\connection.py", line 216, in recv_bytes
buf = self._recv_bytes(maxlength)
File "C:\test\lib\multiprocessing\connection.py", line 306, in _recv_bytes
[ov.event], False, INFINITE)