lacmus-foundation/lacmus

Lacmus is a cross-platform application that helps to find people who are lost in the forest using computer vision and neural networks.

lacmus

The program for searching through photos from the air of lost people in the forest using Retina Net neural nwtwork.

The project is being developed by a non-profit organization Liza Alert.

Demonstration

Picture 1

Picture 2

Video 1

See more examples.

Training data

You can download Lacmus Drone Dataset (LaDD) from mail.ru cloud

• https://cloud.mail.ru/public/2k53/2bJVwYSa7

You also can download Lacmus version of Stenford Drone Dataset (SDD) from mail.ru cloud

• https://cloud.mail.ru/public/4GKW/3FW26Sq77

Usage

Read more about training steps and atraining data at train documentation to learn how to train the model.

Pretrained models

The models are avalable here.

Partners

lacmus's Issues

Makefile ссылается на несуществующие dockerfil'ы

Похоже это следствие изменения структуры при отделении десктопного приложения

docker image(GPU) failed to built

Hi!
i've tried to build GPU-version of docker image on my ubuntu 16.04(nvidia 418.67, Cuda 10.1 and got this error in the end.

sudo docker build --file Dockerfile.gpu -t rescuer_la .
Sending build context to Docker daemon 14.56MB
Step 1/24 : FROM tensorflow/tensorflow:1.12.0-gpu-py3
---> 413b9533f92a
Step 2/24 : ENV DEBIAN_FRONTEND noninteractive
---> Using cache
---> 8a52f51116f2
Step 3/24 : RUN apt-get update -qq && apt-get install --no-install-recommends -y build-essential g++ git wget apt-transport-https curl cython libopenblas-base python3-numpy python3-scipy python3-h5py python3-yaml python3-pydot && apt-get clean && rm -rf /var/lib/apt/lists/*
---> Using cache
---> a545cb38439e
Step 4/24 : RUN pip3 --no-cache-dir install -U numpy==1.13.3
---> Using cache
---> 98d345ea0a28
Step 5/24 : ARG KERAS_VERSION=2.2.4
---> Using cache
---> 7b09457df232
Step 6/24 : ENV KERAS_BACKEND=tensorflow
---> Using cache
---> 85a448c8d80b
Step 7/24 : RUN pip3 --no-cache-dir install --no-dependencies git+https://github.com/fchollet/keras.git@${KERAS_VERSION}
---> Using cache
---> 1c91dbe3620b
Step 8/24 : RUN python3 -c "import tensorflow; print(tensorflow.version)" && dpkg-query -l > /dpkg-query-l.txt && pip3 freeze > /pip3-freeze.txt
---> Running in 08e7f4469a36
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/usr/lib/python3.5/imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.5/dist-packages/tensorflow/init.py", line 24, in
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/init.py", line 49, in
from tensorflow.python import pywrap_tensorflow
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/usr/lib/python3.5/imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/errors

for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
The command '/bin/sh -c python3 -c "import tensorflow; print(tensorflow.version)" && dpkg-query -l > /dpkg-query-l.txt && pip3 freeze > /pip3-freeze.txt' returned a non-zero code: 1

Why getting image sizes using cv2?

There was correction of getting image aspect ration in the PascalVocGenerator, changing PIL.Image.open() usage to cv2.imread:

lacmus/keras_retinanet/preprocessing/pascal_voc.py

Lines 117 to 125 in 09d569a

	def image_aspect_ratio(self, image_index):
	""" Compute the aspect ratio for an image with image_index.
	"""
	path = os.path.join(self.data_dir, 'JPEGImages', self.image_names[image_index] + self.image_extension)
	#image = Image.open(path)
	#return float(image.width) / float(image.height)
	img = cv2.imread(path)
	height, width, _ = img.shape
	return float(width) / float(height)

Why it is considered more performant? As I can see, PIL.Image.open() is more efficient, as it doesn't load whole image in the memory:

https://stackoverflow.com/questions/15800704/get-image-size-without-loading-image-into-memory

I've run an experiment on my machine:

import cv2
from PIL import Image
import os
import time

dir = 'lacmus-foundation/data/laddv4/spring/JPEGImages/'
files = list(os.listdir(dir))

ratio = 0.0
start = time.time()
for f in files:
    image = Image.open(dir + f)
    ratio += float(image.width) / float(image.height)
end = time.time()
print('Getting images sizes from PIL: ', (end - start) / len(files), 'ms per image')

start = time.time()
for f in files:
    img = cv2.imread(dir + f)
    height, width, _ = img.shape
    ratio += float(width) / float(height)
end = time.time()
print('Getting images sizes from cv2: ', (end - start) / len(files), 'ms per image')

It also showed that PIL is quicker:

Getting images sizes from PIL: 0.0014222949298460092 ms per image
Getting images sizes from cv2: 0.3373869177120835 ms per image

Пользовательская документация - работа с данными

• Добавить в вики инструкцию по тому как добавлять данные в проект и как отправлять их на сервер.

• Добавить инструкцию с тем как снимть данные для операторов БПЛА с перечнем поз.

Add Cutmix

Add cutmix generator for beter results.
The cutmix generator can be useful for training models and achieving better results. This will help take existing models to a higher level of quality.

Resources that may be useful:

• https://arxiv.org/abs/1905.04899 - the original article
• https://github.com/clovaai/CutMix-PyTorch - a pytorch implementation
• https://github.com/DevBruce/CutMixImageDataGenerator_For_Keras - keras implenentation (not compartable with retinanet)

can you list out requirement for tensorflow and keras?

what exactly version does it need?

i am hitting all sort of issues.

Typo: Wrong app name in usage in releases' info

Describe the bug
The section "usage" in instructions for releases contains wrong name of the RescuerLaApp. It is named "ReacuerLaApp" for both Linux\CentOS\OSX and Windows. That can cause problems for those who aren't familiar with the project.

There are also a pair if minor typos:

Feature: add function "load and predict"

Для обработки большого количества фотографий нужна дополнителная опция - "загрузить и обработать".

Fix readme logo and organization logo

• Удалить лого Лиза Алерт из ридми.
• Добавить лого lacmus
• Выкинуть лого Лиза Алерт из организации lacmus foundation и заменить его на lacmus
• добавить секцию partners в readme
• добавить лого DTL, Сбер.Клауд, Liza Alert в партнёры.

Feat: add fucnction to reset image back to 100% size

Пилоты очень просили доп. опцию - центрирования и возврата обработанной фотографии к 100%, т.к. при работе на трекпаде фото может "улетать" за пределы галактики.

Thread safe issue with visual_effect_generator

Bug description
The visual_effect_generator, as all python generators, is not thread-safe. That can cause an exception when using several worker threads, especially within single process.

How to reproduce
Enter the lacmus directory and run train.py with --workers > 1 but without --multiprocessing:

keras_retinanet/bin/train.py --backbone mobilenet_v3_small --no-snapshots --batch-size 8 --max-queue-size=10 --workers=8 --epoch 1 --steps 200 pascal ../../../data/ful

Actual result
On one or another training step the process stops with an exception "ValueError: generator already executing"

Callstack:
Traceback (most recent call last):
File "keras_retinanet/bin/train.py", line 546, in
main()
File "keras_retinanet/bin/train.py", line 541, in main
initial_epoch=args.initial_epoch
File "/home/jupyter-kseniia/.local/lib/python3.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/jupyter-kseniia/.local/lib/python3.7/site-packages/keras/engine/training.py", line 1732, in fit_generator
initial_epoch=initial_epoch)
File "/home/jupyter-kseniia/.local/lib/python3.7/site-packages/keras/engine/training_generator.py", line 185, in fit_generator
generator_output = next(output_generator)
File "/home/jupyter-kseniia/.local/lib/python3.7/site-packages/keras/utils/data_utils.py", line 625, in get
six.reraise(*sys.exc_info())
File "/home/jupyter-kseniia/.conda/envs/lacmus-k/lib/python3.7/site-packages/six.py", line 703, in reraise
raise value
File "/home/jupyter-kseniia/.local/lib/python3.7/site-packages/keras/utils/data_utils.py", line 610, in get
inputs = future.get(timeout=30)
File "/home/jupyter-kseniia/.conda/envs/lacmus-k/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
File "/home/jupyter-kseniia/.conda/envs/lacmus-k/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/home/jupyter-kseniia/.local/lib/python3.7/site-packages/keras/utils/data_utils.py", line 406, in get_index
return _SHARED_SEQUENCES[uid][i]
File "keras_retinanet/bin/../../keras_retinanet/preprocessing/generator.py", line 375, in getitem
inputs, targets = self.compute_input_output(group)
File "keras_retinanet/bin/../../keras_retinanet/preprocessing/generator.py", line 347, in compute_input_output
image_group, annotations_group = self.random_visual_effect_group(image_group, annotations_group)
File "keras_retinanet/bin/../../keras_retinanet/preprocessing/generator.py", line 212, in random_visual_effect_group
image_group[index], annotations_group[index]
File "keras_retinanet/bin/../../keras_retinanet/preprocessing/generator.py", line 195, in random_visual_effect_group_entry
visual_effect = next(self.visual_effect_generator)
ValueError: generator already executing
terminate called without an active exception
terminate called recursively
Aborted (core dumped)

Detections are of shape `(1, 200700, 6)` but decode_openvino_detections uses the wrong number of dims

lacmus/cli_inference_openvino.py

Line 57 in f1dd0da

detections = detections[:,:,detections[:,:,:,2].argsort()[0][0][::-1],:] # sort detections on score

As seen here this is assuming 4 dims when output has 3.

Bug: Could not find registration proxy for IID: {ADD8BA80-002B-8F0F-00C04FD062}

Describe the bug
Version - 0.3.2
os - Win10

при попытке загрузить директорию с файлами через USB выдает ошибку
Сообщение программы:

Error Интерфейс не зарегистрирован
Не удалось найти регистрацию прокси-сервера для IID: {ADD8BA80-002B-8F0F-00C04FD062}

Feature: add ability to save picturre with bounding box in a single file.

Для отправки в регионы после обработки фотографий нужно сохранять фото с отметкой в едином файле.

Bug: Application freezes while loading a large number of images (Windows only)

Describe the bug
If you upload more than 200 pictures to Windows, then the application with the operating system sag. ONLY WINDOWS

To Reproduce
Steps to reproduce the behavior:

1. Go to File -> Open directory
2. Select directory with many images
3. App will freeze.

Desktop (please complete the following information):

• OS: Windows 10 x64
• CPU: Intel Core i7-6500U
• GPU: GeForce GTX 950M

Additional context
This bug into avalonia UI 0.8.X. Waiting of 0.9.0 release

Is there a script that converts the keras model to openvino ?

Bug: System.NullReferenceException throws while loading file

Version - 0.3.2
OS - Linux/Windows

На последнем и предпоследнем релизе есть плавающий глюк, систематику еще не выявил. При попытке загрузить с жесткого диска файлы программа падает с таким сообщением:

 Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at RescuerLaApp.Models.Frame.<>c__DisplayClass38_0.<Load>b__0(Object o)
   in /home/user/files/projects/lacmus/RescuerLaApp/Models/Frame.cs:line 77
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

От количества файлов это не зависит, битых файлов нет.

docker image(CPU) failed to start.

Hi, i`ve successfully built CPU image, but it failed to start with this error.

sudo docker run --rm -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix$DISPLAY --workdir=$(pwd) --volume="/home/$USER:/home/$USER" --volume="/etc/group:/etc/group:ro" --volume="/etc/passwd:/etc/passwd:ro" --volume="/etc/shadow:/etc/shadow:ro" --volume="/etc/sudoers.d:/etc/sudoers.d:ro" rescuer_la
No protocol specified
No protocol specified

Unhandled Exception: System.Exception: XOpenDisplay failed
at Avalonia.X11.AvaloniaX11Platform.Initialize(X11PlatformOptions options)
at Avalonia.Controls.AppBuilderBase1.Setup() at Avalonia.Controls.AppBuilderBase1.Start[TMainWindow](Func`1 dataContextProvider)
at RescuerLaApp.Program.Main(String[] args) in /app/install/RescuerLaApp/Program.cs:line 14

add requirements.txt

add requirements.txt with dependencies

Testing script

Do we need testing script for estimating precision metrics and inference time for different models, packed in docker containers? Can I commit it?

Add resnet-like backbones rom segmentation models

fizyr/keras-retinanet#1429

CUDNN_STATUS_INTERNAL_ERROR

CUDNN_STATUS_INTERNAL_ERROR while loading the model

2021-04-05 20:44:46.086918: E tensorflow/stream_executor/cuda/cuda_dnn.cc:328] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
2021-04-05 20:44:46.087682: E tensorflow/stream_executor/cuda/cuda_dnn.cc:328] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
[2021-04-05 20:44:46,090] ERROR in app: Exception on /image [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/dist-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "inference.py", line 132, in predict_image
    caption = run_detection_image(request.json['data'])
  File "inference.py", line 49, in run_detection_image
    boxes, scores, labels = model.predict_on_batch(np.expand_dims(image, axis=0))
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 1788, in predict_on_batch
    outputs = predict_function(iterator)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 780, in call
    result = self._call(*args, **kwds)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 814, in _call
    results = self._stateful_fn(*args, **kwds)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 2829, in call
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 1848, in _filtered_call
    cancellation_manager=cancellation_manager)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 1924, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 550, in call
    ctx=ctx)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
  (0) Unknown:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
     [[node retinanet-bbox/conv1/Conv2D (defined at inference.py:49) ]]
     [[retinanet-bbox/filtered_detections/map/while/body/_1/retinanet-bbox/filtered_detections/map/while/strided_slice_2/_32]]
  (1) Unknown:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
     [[node retinanet-bbox/conv1/Conv2D (defined at inference.py:49) ]]
0 successful operations.
0 derived errors ignored. [Op:__inference_predict_function_7071]

Function call stack:
predict_function -> predict_function

Mon Apr  5 23:06:13 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.102.04   Driver Version: 450.102.04   CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce MX230       Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   64C    P3    N/A /  N/A |    218MiB /  2002MiB |     22%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       956      G   /usr/lib/xorg/Xorg                 95MiB |
|    0   N/A  N/A      1301      G   /usr/bin/gnome-shell              121MiB |
+-----------------------------------------------------------------------------+

docker version: Docker version 19.03.8, build afacb8b7f0

Bug: Program crash with exit code 134

Describe the bug
Program crashes with 134 status code when material.avalonia theme is installed.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'file - open directory'
2. Click on 'predict all' button
3. Open another directory with images (file - open directory)
4. Exuted with code 134

Desktop (please complete the following information):

• OS: Ubuntu 19.04
• CPU: Intel Core i7-6500U
• GPU: GeForce GTX 950M

Additional context
It seems to me that this is due to incorrect visualization errors at material.avalonia theme.