matiasdelellis/facerecognition

Nextcloud app that implement a basic facial recognition system.

Face Recognition

Nextcloud app that implement a basic facial recognition system.

FaceRecognition is a Nextcloud application with a goal of recognizing, analyzing and aggregating face data in users images, and providing additional functionalities on top of these information, all with built-in privacy of Nextcloud. Imagine Google Photos, but only for faces (not detecting objects…) and in such way that your images never leave your Nextcloud instance. 😃

The application listens to the creation of new image files, and queues them for later analysis. A scheduled task (Or admin on demand) take this queue, and analyze the images for looking faces and if possible identify them grouping by similarity.

How to use it?

The administrator must properly configure the application, and once it is working, the user must accept that he wants to allow the analysis of his images to discover his friends. Finally the user can use the application in three ways

1. In the user settings there is a 'Face Recognition' panel where first of all each user must enable the analysis. Once enabled, you will progressively see the discovery of your friends, and you can assign them names.
2. In the file application the user can search by typing your friend's name, and it will show all the photos.
3. In the side panel of the file application, a 'Persons' tab is added where you can see a list of your friends in the photo, and rename them. Also you can select the folders you want to ignore for the process.
4. In the side panel of the Photos application, a 'Persons' tab is added where you can see a list of your friends in the photo, and rename them.

Donate

If you'd like to support the creation and maintenance of this software, consider donating.

Installation, configuration and usage

Requirements

• Nextcloud 22+
• Dlib PHP bindings
• PHP Bzip2
• 1GB of RAM

Installation

Ideally once you meet the requirements, you can install and enable it from the nextcloud app store. For details and advanced information read the documentation about installation.

Configuration

Before proceeding to analyze the images, you must indicate how much memory you want to assigns to image processing and then must properly install and configure the pretrained models using the occ face:setup command. For details and advanced information read the documentation about models.

Then you must indicate the size of the images used in the temporary files from the Nextcloud settings panel. This configuration will depend on your installation and has a direct impact on memory consumption. For details and advanced information read the documentation about Temporary files.

Test the application

We recommend test the application intensively before proceeding to analyze the real data of the users. For this you can create a new user in your Nextcloud instance and upload some photos from the internet. Then you must run the occ face:background_job -u new_user -t 900 command for this user and evaluate the result. For details and advanced information read the documentation of this command below.

Schedule background job

The application is designed to run as a scheduled task. This allows analyze the photos and showing the results to the user progressively. You can read about some ways to configure it within our documentation about Schedule Background Task.

occ commands

The application add commands to the Nexcloud's command-line interface.

Initial setup

occ face:setup [-M|--memory MEMORY] [-m|--model MODEL]

This command is responsible for making the necessary settings to use the application.

If MEMORY is supplied, it will establish the maximum memory to be used for the processing of the images. This value will be limited according to the memory available by the system and the php configuration. You can use numbers as bytes (1073741824 for 1GB), or subfixed with units (1024M or 1G) but note that it is without space.

If MODEL_ID is supplied. the pre-trained model for facial recognition will be installed.

You must perform both settings before continuing with any application command. If you do not supply any of these options, the command will return the current configuration.

Face analysis

occ face:background_job [-u|--user_id USER_ID] [-t|--timeout TIMEOUT] [--defer-clustering] [-M|--max_image_area MAX_IMAGE_AREA]

This command will do all the work. It is responsible for searching the images, analyzing them and clustering faces found in them in groups of similar people.

Beware that this command can take a lot of CPU and memory! Before you put it to cron job, it is advised to try it out manually first, just to be sure you have all requirements and you have enough resources on your machine.

Command is designed to be run continuously, so you will want to schedule it with cron to be executed every once in a while, together with a specified timeout. It can be run every 15 minutes with timeout of -t 900 (so, it will stop itself automatically after 15 minutes and cron will start it again), or once a day with timeout of 2 hours, like -t 7200.

If USER_ID is supplied, it will just loop over files of a given user. Keep in mind that each user must enable the analysis individually, and otherwise this command will ignore the user.

If TIMEOUT is supplied it will stop after the indicated seconds, and continue in the next execution. Use this value in conjunction with the times of the scheduled task to distribute the system load during the day.

If MAX_IMAGE_AREA is supplied caps the maximum area (in pixels^2) of the image to be fed to neural network, effectively lowering needed memory. Use this if face detection crashes randomly.

If use the --defer-clustering option, it changes the order of execution of the process deferring the face clustering at the end of the analysis to get persons in a simple execution of the command.

Create albums in the Photos app

face:sync-albums [-u|--user_id USER_ID]

This command creates photo albums within the Nexcloud Photos app, with photos of each person found.

Note that these albums are editable in the Photos app, and any changes will be ignored and reverted on the next run of this command.

This command is also designed to be run regularly to sync any user changes, as this command is the only one that will update albums.

If USER_ID is provided, it will just sync albums for this user.

Resetting information

occ face:reset [--all] [--model] [--image-errors] [--clustering] [-u|--user_id USER_ID]

This command can completely wipe out all images, faces and cluster of persons. It is ideal if you want to start from scratch for any reason.

You must specify if you wish to completely reset the database [--all] or just the current model [--model] and all images must be analyzed again, or or you can reset only the clustering of persons [--clustering] and only clustering needs to be done again, or reset only the images that had errors [--image-errors] to try to analyze them again.

If USER_ID is provided, it will just reset the information of a particular user.

Migrate models

occ face:migrate [-m|--model_id MODEL_ID] [-u|--user_id USER_ID]

This command allows to migrate the faces obtained in a model to a new one. Note that the persons name are not migrated, and the user must rename them again. Always is recommended to analyze from scratch any configured model, but you can save a lot of time migrating it.

You must specify which model you want to migrate using the MODEL_ID option.

If USER_ID is provided, just migrate the faces for the given user.

Statistics

occ face:stats [-u|--user_id USER_ID] [-j|--json]

This command return a summary of the number of images, faces and persons found.

If USER_ID is provided, just return the stats for the given user.

If use the --json argument, it prints the stats in a json format more suitable to parse with other tools.

Progress

occ face:progress

This command just return the progress of the analysis and an estimated time to complete.

If use the --json argument, it prints the stats in a json format more suitable to parse with other tools.

facerecognition's Issues

[Exception] Unknown image file format: Unable to load image in file /tmp/facerec_t5q9Y4.jpg

Hi @stalker314314

Just now I am testing your branch, and seeing what I can contribute. 😄

Well,
I have an image, which seems to open as invalid -Although I can download it and see it well-, and breaks the background task. Resulting in:

yielding
Processing image /media/datos/Services/nextcloud/data/yany7213/files/Documentos/Compartido Matias/Tarjetas/IMG-20170427-WA0014.jpg
OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask::resizeImage(): No image loaded

[Exception]
Unknown image file format: Unable to load image in file /tmp/facerec_t5q9Y4.jpg

Checking your code:

facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php

Line 165 in 73eaccb

return 1.0;

In case of error, it should return -1.0 and then check that ratio is greater than 0.0

On the other hand, IMHO the verification of whether the original image is valid, must be done before calling resize.

facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php

Line 125 in 73eaccb

// todo: be smarter with this 1024 constant. Depending on GPU/memory of the host, this can be larger.

I could do some pull request for this, but for now I prefer that you continue with the background task yourself. 😄
Thanks for all 😃

Make sure temp images are deleted in finally block

In ImageProcessingTask

Release to the app store?

Hello,

this looks like an interesting app. Do you intend to put it on the Nextcloud App Store? That would be awesome.

Handle .nomedia in Watcher

@matiasdelellis added support for .nomedia. It looks like a good feature, but we need to honor it in Watcher.php. Seems that we can accidentally add images from .nomedia folder (or child of it).

App is not compliant

App is not compliant

Just add this report to remember it.

Travis CI checks these reports but deliberately ignores them.

https://github.com/matiasdelellis/facerecognition/blob/master/.travis.yml#L32

Test

[matias@nube nextcloud]$ sudo -u apache php occ app:check-code  facerecognition
[sudo] password for matias: 
App is not compliant

Details:

OC_Util - Static method of private class must not be called

Analysing /var/www/html/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/AddMissingImagesTask.php
 2 errors
    line  123: OC_Util - Static method of private class must not be called
    line  124: OC_Util - Static method of private class must not be called
Analysing /var/www/html/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/StaleImagesRemovalTask.php
 2 errors
    line  137: OC_Util - Static method of private class must not be called
    line  138: OC_Util - Static method of private class must not be called
Analysing /var/www/html/nextcloud/apps/facerecognition/appinfo/app.php
 1 errors
    line   31: OC_Util - Static method of private class must not be called

Private API is used, therefore it is valid, however there is no replacement with public api. So, ignore this.

Fixed

Fixed on PR #196

Database schema error: Name of table dbprefixface_recognition... is too long:

Database schema error: Name of table *dbprefix*face_recognition_face_models is too long (28), max. 27 characters (21 characters for tables with autoincrement) + *dbprefix* allowed
Database schema error: Name of table *dbprefix*face_recognition_face_models is too long (28), max. 27 characters (21 characters for tables with autoincrement) + *dbprefix* allowed
Database schema error: Name of table *dbprefix*face_recognition_persons is too long (24), max. 27 characters (21 characters for tables with autoincrement) + *dbprefix* allowed
Database schema error: Name of table *dbprefix*face_recognition_images is too long (23), max. 27 characters (21 characters for tables with autoincrement) + *dbprefix* allowed
Database schema error: Name of table *dbprefix*face_recognition_faces is too long (22), max. 27 characters (21 characters for tables with autoincrement) + *dbprefix* allowed

Although we agree that long names are not recommended, we believe that it is not a problem.

OC_DB - Static method of private class must not be called

Analysing /var/www/html/nextcloud/apps/facerecognition/appinfo/update.php
 1 errors
    line   14: OC_DB - Static method of private class must not be called
Deprecated file found: /var/www/html/nextcloud/apps/facerecognition/appinfo/update.php - please use repair steps

It was a quick table migration that will be unnecessary after publishing the application, and will be directly deleted.

Fixed on PR #172

Invalid appinfo.xml file found: Element 'types': This element is not expected

Invalid appinfo.xml file found: Element 'types': This element is not expected. Expected is one of ( background-jobs, repair-steps, two-factor-providers, commands, settings, activity, dashboard, fulltextsearch, navigations, contactsmenu ).

Although it is not documented, it is necessary. So, ignore this.

• #73
• nextcloud/server#12942

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

PersonMapper is wrom?

See:

https://github.com/matiasdelellis/facerecognition/blob/master/lib/Db/PersonMapper.php#L38

It is associated with the Image entity.. 😕

Face recognition app is not showing any faces

I have Nextcloud 14. Latest version from master of face recognition.

So, I am visiting https://[site]/index.php/settings/user/facerecognition. After some loading, this is what I get:

When I hover over thumbnails, I get this link https://[site]/f/35494 and when I click on that thumbnail, I get this:

In network tab (Firefox web developer tools), I see bunch of requests going to https://[site]/apps/facerecognition/face/198/thumb/50, but they all finish with 404.

Basically, what I need to do/tweak/install to see thumbnails?:)

Add task to remove images that do not exist physically, but are in DB

This task will be needed only if user adds file .nomedia in some folder and all images in that folder and in child folders should be removed from our DB!

But, instead of doing this in hook (it could be very, very expensive), my proposal is to do it in new task - something like RemoveMissingImages. This task should take all images from DB, check that they actually exist (and no parent above them have .nomedia) and remove them from DB if this is not true (using same logic as delete hook in Watcher.php). This task should be guarded with some config value (similar to AddMissingImagesTask::FULL_IMAGE_SCAN_DONE_KEY), where default value should be false, meaning this task is not run, unless postWritev2 detects that .nomedia was added in some folder and set this to true, so next background job will execute this.

CropFace is loading same image two times for no reason (perf issue)

In ImageProcessingTask

Decide on scaling image

Today, we scale image to 1024px (1024px is larger dimension!), with upsampling set to 0. This is mostly value that is set because it worked on my machine😄. Going forward, I wanted to see what is really this value. Higher value means more RAM is needed. Lower value means small faces will not be detected. So, I did some analysis and published it on my blog:
https://blog.kokanovic.org/performance-analysis-of-dlibs-cnn-face-detection/

@matiasdelellis - please read when you find time (it is lot of text😄). Basically, some questions for you:

• Are you OK to move content from blog post to wiki here?
• Are you OK to commit python code, output, produced images to repo here (let's say directory "analysis/" in root?)
• Are you OK with hardcoding upsampling to 0 and changing scaling based on RAM of user? I would go with something like this:
  • Read total physical memory on machine
  • If it is less than 2GB and we don't allow usage of Face Recognition app (I really cannot imagine how it can be usable with less than that:/ And even that is bare minimum)
  • Assume that free/available memory is 1/2 of total RAM memory (I would not go into inpecting free -h results (or amount of swap space) and use that in calculations, as it changes over time and we want consistent budget over time (unless user upgrades RAM physically😄)
  • Formula: Scale [megapixels] = RAM[GB] / 1000 (so, 2000GB of RAM will scale to 2MP, 4000GB to 4MP...)

Again, I would not go into inspecting currently free memory, because scaling will be very dependent on current load on machine. Also, I would not go into some self-adaptive logic (for example, if we hit OOM, we try again with lower size...)

Your thoughts?

Choose file type we are supporting

We have in code sprinkled support for image/jpeg and/or image/png. Some questions:

• should we decide file type based on mime type from Nextcloud? Probably yes
• what mimetypes Nextcloud can handle?
• what types dlib handles?
• what happens if we add support for more filetypes? Should we kick in new AddMissingImagesTask run? Reset FULL_IMAGE_SCAN_DONE_KEY in migration?

Do not do face recognition for small images

I got bunch of small files in my instance (from random programs, games, books, backups, I don't even know from where:)). They are not in my "main" directory of photos. I do not want recognition on them, because it will take needless time, needless DB space and will add noise to results. I think proper solution might be having whitelist/blacklist of folders per user, but it adds another layers of complexity. I think what we can do now is to ignore small files. Let's say - all files:

• with less than 0.5MP (megapixels), or
• width < 500px, or
• height < 500px
  are most likely not coming from digital camera and we should be ignoring them. This should be configurable though, so advanced users can set this config.

Rewrite FULL_IMAGE_SCAN_DONE_KEY to be per-user, not per-instance

When we reset FULL_IMAGE_SCAN_DONE_KEY, there is no reason to do full scan of all images of all users. Only affected users' images should be rescaned. This will also help logic from PR #84 where going off->on should trigger rescan.

Personal => Face Recognition page result in error: the Uncaught EvalError: Refused to evaluate a string as JavaScript

I just installed the app manually on my system:

• Ubuntu 16.04
• Nextcloud 15

And when opening Personal => Face Recognition in the browser (Chrome 71), I see this error in the console:

Uncaught EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval' 
is not an allowed source of script in the following Content Security Policy directive:
"script-src 'nonce-d0FNdFhKT3pqc2F4NHRsWWV0UERhclVPb3FHYVdyNmVOdWE4U1g3N2NBaz06aGxWQUphRGR2NEhEcHZJTU1iaUdDUFpoODVueWFkZmtYYmFNY0N5Q0NWcz0='".

    at JavaScriptCompiler.Function (<anonymous>)
    at JavaScriptCompiler.createFunctionContext (handlebars.js?v=27e5d68a-4:3562)
    at JavaScriptCompiler.compile (handlebars.js?v=27e5d68a-4:3450)
    at JavaScriptCompiler.compileChildren (handlebars.js?v=27e5d68a-4:4102)
    at JavaScriptCompiler.compile (handlebars.js?v=27e5d68a-4:3405)
    at JavaScriptCompiler.compileChildren (handlebars.js?v=27e5d68a-4:4102)
    at JavaScriptCompiler.compile (handlebars.js?v=27e5d68a-4:3405)
    at JavaScriptCompiler.compileChildren (handlebars.js?v=27e5d68a-4:4102)
    at JavaScriptCompiler.compile (handlebars.js?v=27e5d68a-4:3405)
    at compileInput (handlebars.js?v=27e5d68a-4:3249)

And way to fix it?

Continuous integration support

Add support for Travis. With existing tests. Code coverage is bonus:)

Add composer.json

There are bunch of error in Scrutinizer that some import is missing and that they can be fixed if we have composer.json. I am talking about random stuff like this. I noticed other apps actually have this file. Other than that, I do not see (currently) reason to have it, there is no dependencies for this project:) However, I guess it would be nice to have it...one day:)

Cleanup after user is deleted

When user is deleted, we should remove all persons, images and faces associated with this user. This can be implemented either as a hook or part of background job. I guess hook will block more, but we can detect issues better with this approach:)

Keep 128d vector in DB

For each analysed face/image. It seems more flexible and you can reuse it later (for chinese whispers) and know your progress on image processing? What do you think?

Don't allow background jobs from AJAX cron jobs

(TODO in BackgroundService.php)

We should disallow AJAX cron jobs to be able to run background jobs. This defies whole purpose of background job (notice what is highlighted:smile:) But, somewhere in admin settings, there should be warning telling admin that background server (and therefore whole app) is not working.

Support for Nextcloud 14.0

I just wanted to test this app but unfortunately it only works with Version 13 or lower. Error: Server version 13 or lower is required.

Check if we should rely on file cache in DB tables

Currently, in face_recognition_images table, we have column file pointing to filecache entry. We need to check if this is good idea, or, alternatelyvey we should rely on string (path to image)

Reimplement choosing temp folder logic

Currently, we use machine's temp folder to store resized versions of images. This is not secure, We need to leverage NC capabilities to have temp folders. @matiasdelellis had comment and also idea here:

#4 (comment)

Add PHP memory recommendation to README

Please add minimum required PHP memory and recommended PHP memory setting to README

Support to link cluster to federated users

linked_user column in face_recognition_persons table should support federated users too. Maybe it already is/will support (out of the box), but check first

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Handle clusters that do not exist anymore

TODO from mergeClusterToDatabase. To be honest, I completely forgot what was this about:)

Decide on minimum Nextcloud version

Data points (add more as we go):

• v14 has requirement on PHP>=7. This removes one dimension when we think about pdlib support
• v14 has QBMapper. Not really that important, but we duplicated frFindEntities functionality which we could remove if we require >=14
• v14 is fairly new and I guess people are still on old version. Not sure is there any way to see adoption

Add progress bar and verbose option to background task.

Now it is very difficult to know the progress of the background task..

I suggest showing a numerical progress, and add a vervose option, which would work as it is now...

https://symfony.com/doc/current/components/console/helpers/progressbar.html

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

PDlib debian packages

Maybe I should add this issue in PDLib repository, but I find it much better to add it here, ha ha. Assigned to you @matiasdelellis (feel free to remove assignee label:D). I think this is blocker for beta version, as this can help with installations. This can also help to be written in FAQ and it will unblock bunch of integration tests.

BTW, where will those debian packages be hosted? LMK if you need some VMs or any other help for this task!

Per-directory settings support

@matiasdelellis thought of this idea, let's use this issue as placeholder for discussion. Idea is to have support for per-directory settings, with new file, maybe .nofacemedia, .facerecognition.json, or .facerec_settings.json where user could put some settings that apply for that directory only. Here are some ideas what we could put there:

• face recognition on/off,
• filter for files,
• file types (maybe same as above?)
• min image size
• is it recursive or just applies to current directory (not sure how to model this)

This is definitively not something for v1, but just throwing ideas. Maybe syntax could be:

{
    "detection: "on",
    "min_image_size": 1024,
    "file_types": ["jpg", "bmp"],
    "included_filter": ["*family/*", "*/work"],
    "excluded_filter": ["*/family/videos", "*HDR"]
}

I don't know if there is a need for this, maybe is someone asks (for example, I don't think I need this level of granularity...yet:)

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Checks if are using GPU

Not sure how we can know this, I think we will need to drill some method in pdlib to get to DLib, maybe DLib can tell us:

• if it is compiled with CUDA support, and
• If there is CUDA GPU present

After we know that, some calculation on image size can be different

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Handle case when in postWrite file changes mimeType

In postWritev2, if file was JPEG, but was overwritten with, let's say .txt, we have problems. Also handle other way around (.txt->.jpg)

Central place for loading DNN models

In ImageProcessingTask class, there are loading of models, which should be better if they are in Requirements class, or somewhere else, central to where we can find it (with more complex lookup logic to find models). This task also depends on how we load and keep models (apt-get/manually, or automaticall in future)

The hooks are not working

At least in my case, the storage is reporting as Trash, and as we only read the LOCAL storage the hooks not work.

How to test?

diff --git a/lib/Watcher.php b/lib/Watcher.php
index 1a3c927..be7170c 100644
--- a/lib/Watcher.php
+++ b/lib/Watcher.php
@@ -101,6 +101,8 @@ class Watcher {
        public function postWrite(Node $node) {
                $model = intval($this->config->getAppValue('facerecognition', 'model', AddDefaultFaceModel::DEFAULT_FACE_MODEL_ID));
 
+               $this->logger->debug("CLASS: 0: ".get_class($node->getStorage()));
+
                // todo: should we also care about this too: instanceOfStorage(ISharedStorage::class);
                if ($node->getStorage()->instanceOfStorage(IHomeStorage::class) === false) {
                        return;

In my case it results in:

CLASS: 0:OCA\Files_Trashbin\Storage

😕 😞

Allow user to specify used memory

We can have some setting (not per person, it needs to be admin setting) to allow admin to choose amount of memory for our app to use. This will, in turn, affect image size that we scale image to, and that means more faces that can be recognized. This should be some advanced setting. It should be in [1GB, memory_limit] range. Use cases:

• User have memory_limit to 8GB and do not want PHP to use whole 8GB, but only 4GB
• User have memory_limit to -1, and system RAM to 4GB and do not want our app to use 2GB, but 3GB.

Adaptive image resize based on GPU memory

Currently, in ImageProcessingTask, we resize image to 1024 as a "good enough" way not to overload GPU with large image. Depending on whether we are using CPU or GPU dlib implementation, and on amount of RAM and GPU memory of host machine, we can increase/decrease this value. Of course, we first have to obtain these parameters and to know proper values for thise magic 1024 constant.

Two problems here:

• quality of face recognition can vary if user changed dlib implementation, or add more RAM... and we want to be consistent, at least:)
• if we add this later in app lifetime, people might get better/worse results from face recognition.

Not sure how much this scaling is actually contributing to face recognition quality, might not that much and problems stated above do not really exists, so we need more data on these

Test if we are resilient to OOM during face recognition

And what to do here? See if PHP just crashes immediately and we cannot do anything or we can catch some exception from dlib? If we can catch it and act on it, what should we do? Skip image and go to next one? I don't think so, as I think all images will fail in similar fashion, so we need to stop. Also, how to notify user on this, as obviously it doesn't have enough memory. We will need to have some notification in settings/user/facerecognition, but backend needs to know that last iterations ended with OOM, not sure how to preserve that info (in config?). @matiasdelellis - I am interested in your ideas here?

There is no clustering until I analyze all the photos of all the users.

I just want to enable a user for tests but until I analyze all the users, grouping is not done... 😞

On the other hand the task CreateClustersTask is executed before ImageProcessingTask, therefore, all the photos must be analyzed, and then launch it again to do the grouping.. 😉

Support to give specific subdirectory path and files when running command

Say, user wants to process just some subdirectory, it would be good if we can support

./occ face:background_job --path /files/username/pics/just_one_vacation

It will affect some tasks (where applicable), but some will be indeferent to this.

Of course this must accept files as parameters, to allow analysis of point images, which allows testing. Issue #245

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Support for shared storage

Watcher.php has some TODO for this (ISharedStorage). But yeah, in general, we want to know are we shared-storage friendly, or not.

Throw exception in LockTask if lock is already held

Check Doctrine JSON type with MySQL and SQLite

Currently, face_recognition_faces DB table has descriptor column of JSON type, which have json_array type. Double check that this can be used for MySQL and SQLite too. See also related issue #11 (if #11 is done, maybe this becomes non-issue)

Support for HOG detector

Current (CNN) detector is super precise and super slow:) HOG is very fast, but it can also find face in every lamp-post:) Still, it is...usable. Friend was telling me recently that he used HOG for fast and dirty face clustering (and he says all lamp posts were clustered together:D). Anyway, that got me thinking - we now require 2GB and 60 seconds per image. We should be able to offer low-quality, poor-result, bu available-for-everyone option. I think HOG is the detector that you @matiasdelellis used when you had Python (you used get_frontal_face_detector)?
I guess that can increase amount of people that can use this app. And I guess it shouldn't be that hard to include it - just add new face_model (id=2), and add additional model files and put if (model==2) to couple of places:) So, @matiasdelellis - back to the origins, what do you say?:) (I think this might not be a blocker for some initial beta release)

Lazy caching to get user folders in ImageProcessingTask

There is big giant for loop in ImageProcessingTask and it picks random image from random user (this is all by design) and it tries to load user folder. 1) Check if this hits disk/IO, and if so, 2) implement some hashmap logic with some lazy loading of user folders (rationale is that we will probably have more images than users and we will "jump" between users, so better have user folder cached)

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Log messages in FaceRecognitionLogger even neither output nor logger is not set

This should never happen. Defensive programmer in me tells me we should raise exception here:) Not sure, though.

Exception handling in ImageProcessingTask

Number of things that can go wrong in this logic is just too damn high.

https://knowyourmeme.com/memes/the-rent-is-too-damn-high-jimmy-mcmillan

Drop database as test, and found at lest two bugs.

Drop all my faces tables as test and launch the background task. The analysis is done completely, but there is no clusters.

On log see these:

A non well formed numeric value encountered at /var/www/html/nextcloud/apps/facerecognition/lib/Helper/MemoryLimits.php#105

https://github.com/matiasdelellis/facerecognition/blob/master/lib/Helper/MemoryLimits.php#L105

And:

Division by zero at /var/www/html/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/CreateClustersTask.php#167

https://github.com/matiasdelellis/facerecognition/blob/master/lib/BackgroundJob/Tasks/CreateClustersTask.php#L167

In the database now observed that:

• oc_face_recognition_persons is empty.
• oc_face_recognition_faces is empty.
• oc_face_recognition_images has all the files, but with error 'std::bad_alloc'.

resizeImage in ImageProcessingTask should throw exception on error

Can we just delete AddDefaultFaceModel post-migration steps

We currently have AddDefaultFaceModel post-migration step which only does one thing - adds default model with ID=1. Maybe it can be part of installation step. What if we delete app and install it again?

RegisterAdmin call is deprecated, create proper settings

Current approach is to create admin page with:

\OCP\App::registerAdmin

and this is deprecated:

@deprecated 14.0.0 Use settings section in appinfo.xml to register admin sections

This is blocking ./occ app:check to work in continuous integration

I guess we need to rewrite admin page to be in new (NC>=v10) format.

Compress descriptor in database

face_recognition_faces DB table has descriptor column which is plain raw JSON. It contains array of 128 float numbers. We should probably have better representation for this array. We don't do any calculation with it in DB, only in business logic, so it can be compressed better. See also related issue #10.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.