jakewilliami/facedetection.jl

A face detection algorithm using Viola-Jones' rapid object detection framework written in Julia

FaceDetection using Viola-Jones' Robust Algorithm for Object Detection

---

Caveats;— help wanted

Currently, there are two main areas I would love some help with:

1. Reading from pre-trained data files would greatly help those who are not training the object detection framework themselves; and
2. Creating (scalable) bounding boxes around detected objects. Everyone loves a visual; you can show your grandmother a face with a box around it, but she'll switch off if you show her any code. People want pictures!

Since this project helped me with the get_faceness function (this idea of scoring the "faceness" of an image), now that I have data from this I have put this project on the back burner. However, in the interest of others, if anyone were to help with these two items (and any other issues), they would forever have my gratitude.

---

Introduction

This is a Julia implementation of Viola-Jones' Object Detection algorithm. Although there is an OpenCV port in Julia, it seems to be ill-maintained. As this algorithm was created for commercial use, there seem to be few widely-used or well-documented implementations of it on GitHub. The implementation this repository is based off is Simon Hohberg's Pythonic repository, as it seems to be well written (and the most starred Python implementation on GitHub, though this is not necessarily a good measure). Julia and Python alike are easy to read and write in — my thinking was that this would be easy enough to replicate in Julia, except for Pythonic classes, where I would have to use structs (or at least easier to replicate from than, for example, C++ or JS — two other highly-starred repositories.).

I implore collaboration. I am an undergraduate student with no formal education in computer science (or computer vision of any form for that matter); I am certain this code can be refined/optimised by better programmers than myself. Please, help me out if you like!

How it works

In an over-simplified manner, the Viola-Jones algorithm has some four stages:

1. Takes an image, converts it into an array of intensity values (i.e., in grey-scale), and constructs an Integral Image, such that for every element in the array, the Integral Image element is the sum of all elements above and to the left of it. This makes calculations easier for step 2.
2. Finds Haar-like Features from Integral Image.
3. There is now a training phase using sets of faces and non-faces. This phase uses something called Adaboost (short for Adaptive Boosting). Boosting is one method of Ensemble Learning. There are other Ensemble Learning methods like Bagging, Stacking, &c.. The differences between Bagging, Boosting, Stacking are:
   • Bagging uses equal weight voting. Trains each model with a random drawn subset of training set.
   • Boosting trains each new model instance to emphasize the training instances that previous models mis-classified. Has better accuracy comparing to bagging, but also tends to overfit.
   • Stacking trains a learning algorithm to combine the predictions of several other learning algorithms. Despite this method being developed at the start of the century, it is blazingly fast compared to some machine learning algorithms, and still widely used.
4. Finally, this algorithm uses Cascading Classifiers to identify faces. (See page 12 of the original paper for the specific cascade).

For a better explanation, read the paper from 2001, or see the Wikipedia page on this algorithm.

Quick Start

using FaceDetection

# Constants
pos_training_path, neg_training_path = "...", "..."
num_faces, num_non_faces = length(filtered_ls(pos_testing_path)), length(filtered_ls(neg_testing_path))  # You can also just put in a simple number here, if you know how many training images you have
num_classifiers = 10
min_feature_height, max_feature_height = 0, 19
min_feature_width, max_feature_width = 0, 19
scale, scale_to = true, (19, 19)  # we want all training images to be standardised to size 19x19

# Train a model
classifiers = FaceDetection.learn(pos_training_path, neg_training_path, num_classifiers, min_feature_height, max_feature_height, min_feature_width, max_feature_width; scale = scale, scale_to = scale_to)

# Results
correct_faces = sum(ensemble_vote_all(pos_testing_path, classifiers, scale=scale, scale_to=scale_to))
correct_non_faces = num_non_faces - sum(ensemble_vote_all(neg_testing_path, classifiers, scale=scale, scale_to=scale_to))
println((correct_faces / num_faces) * 100, "% of faces were recognised as faces")
println((correct_non_faces / num_non_faces) * 100, "% of non-faces were identified as non-faces")

For more examples like this, see examples/.

Miscellaneous Notes

Timeline of Progression

• a79ab6f9 — Began working on the algorithm; mainly figuring out best way to go about this implementation.
• fd5e645c — First "Julia" adaptation of the algorithm; still a lot of bugs to figure out.
• 2fcae630 — Started bug fixing using src/FDA.jl (the main example file).
• f1f5b5ea — Getting along very well with bug fixing (created a struct for Haar-like feature; updated weighting calculations; fixed hstack translation with nested arrays). Added detailed comments on each function.
• a9e10eb4 — First working draft of the algorithm (without image reconstruction)!
• 6b35f6d5 — Finally, the algorithm works as it should. Just enhancements from here on out.
• 854bba32 and 655e0e14 — Implemented facelike scoring and wrote score data to CSV (see #7).
• e7295f8d — Implemented writing training data to file and reading from that data to save computation time.
• e9116987 — Changed to sequential processing.
• 750aa22d–b3aec6b8 — Optimised performance.

Acknowledgements

Thank you to:

• Simon Honberg for the original open-source Python code upon which this repository is largely based. This has provided me with an easy-to-read and clear foundation for the Julia implementation of this algorithm;
• Michael Jones for (along with Tirta Susilo) suggesting the method for a facelike-ness measure;
• Mahdi Rezaei for helping me understand the full process of Viola-Jones' object detection;
• Ying Bi for always being happy to answer questions (which mainly turned out to be a lack of programming knowledge rather than conceptual; also with help from Bing Xue);
• Mr. H. Lockwood and Mr. D. Peck are Comp. Sci. students who have answered a few questions of mine;
• Finally, the people in the Julia slack channel, for dealing with many (probably stupid) questions. Just a few who come to mind: Micket, David Sanders, Eric Forgy, Jakob Nissen, and Roel.

A Note on running on BSD:

The default JuliaPlots backend GR does not provide binaries for FreeBSD. Here's how you can build it from source.. That said, StatsPlots is only a dependency for an example, and not for the main package.

facedetection.jl's Issues

Ensure correctness of faceness measure

This is an adapted feature. I think I slightly misunderstood, and this needs to be corrected.
The faceness measure needs to be reviewed to ensure its correctness. It should be a feature of the cascade, which itself needs to be assessed.

Adjust `getVote` function to see at which point the cascade stops, and return that "face-likeness" score

Probably best to make a separate getVote function for this modification, in case I make this a useable package for people.

Support rotated Haar features

Fix speed of sequential processing

Scale classifiers (to ensure that classifiers work on images different sizes to training data)

Add other Face Detection methods (not just Viola Jones)

This includes adding documentation and giving wider examples of how to use this repo. Make test files and such. I don't know how packages in Julia work...yet!

Read pretrained data from file

Use sparse arrays to save on memory

We do need to consider if sparse arrays are always necessary. E.g., perhaps they might save on memory in our initial construction of the matrices (rather than filling them with zeros), but it might confuse the programme to have them still implemented in the updating/weighting of the features. Need to think about this.

Also need to consider which sparse array format. E.g., CSC (column sparse compressed) format (dump(sparse([0 8 9; 5 0 0; 12 0 4])))?

(Either one of the assignees is able to work on this, as it would not be hard to implement, just a little time consuming).

Add docstrings

Ensure IntegralArray adheres to AbstractArray interface

Rename function names to be in line with Julia coding convention

See a remark on this here.

Failing on 32-bit machines

Optimise memory usage: GPU array storage and other small changes

#24205 in julia lang https://github.com/JuliaLang/julia/issues/

Ensure all `print` functions are inside of FDA.jl (i.e., the main programme)

clean up code after bug fixing

There are far too many unneeded comments

Add code refinement to this repo

• See Simon-Hohberg/Viola-Jones#17.

Make files modules and put exports at the start

Hence, I can use using rather than import.

E.g.,

module foo

using blah: la
using bar

export hi, hello

#=
module code goes here
=#

end # end module

Ensure $\alpha_t=\log\left(\frac{1}{\frac{\varepsilon_t}{1-\varepsilon_t}}\right)$

See commit ac61810.

Be explicit in which modules the functions come from

Extends #15

Zygote integeration

Might be fun to AD through CV algorithms. Related: Check out DiffImages.jl

during code cleanup go through and add `throw` for certain things (to make illegal inputs more readable)

Consider changing deep* functions back to Base functions

The deep* functions are now outdated, after we fixed the array types

Put Haar-like features on image

Perhaps similar to this here. Unsure how to do this at the minute.

Use dict to define feature_types rather than referencing ordered array

This is work towards readability of code

Ensure calculations for weighting is true to the original algorithm

print(weights) at the line before this line will print the array

[0.00020585 0.00020585 0.00020585 ... 0.00020585 0.00020585 0.00020585]

And print(weights) after said line will print arrays

[0.00020585 0.00020585 0.00020585 ... 0.00020585 0.00020585 0.00020585]
[0.00020585 0.00020585 0.00020585 ... 0.00020585 0.00020585 0.00020585]
[0.00013831 0.00013831 0.00013831 ... 0.00013831 0.00030636 0.00030636]

I need to replicate this in Julia.

Print messages in src/FDA.jl with colour

Statistically analyse the differences in box plots.

Ensure all libraries are needed

There may be remaining but redundant libraries loaded from testing at the beginning of this repo

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

Refine getIntegralImage as per help from JuliaImages people

See here.

Determine minFeatureHeight, maxFeatureHeight, minFeatureWidth, and maxFeatureWidth better

e.g.,

minFeatureHeight = 1
minFeatureWidth = 1
maxFeatureHeight = 1
maxFeatureWidth = 1

currentSize = (0, 0)

for picture_directory in list_of_directories
    for picture in picture_directory
        newSize = size(load(joinpath(homedir(), "FaceDetection.jl", "data", picture_directory, picture))
        currentSize = newSize < currentSize ? newSize : currentSize
    end
end

maxFeatureHeight = currentSize[2]
maxFeatureWidth = currentSize[1]

or something like that

Get file name in csv

Fix discrepancy with pythonic results

There is a discrepancy in results of this algorithm compared to the Pythonic one. Both algorithms work, but produce different results.

Ensure `getScore` and `getVote` have strict parameter typing

I had to remove it because I was getting the error

ERROR: LoadError: MethodError: no method matching getVote(::Main.FaceDetection.Adaboost.HaarLikeFeature.HaarLikeObject, ::Array{Float64,2})
Closest candidates are:
  getVote(::Main.FaceDetection.Utils.HaarLikeFeature.HaarLikeObject, ::AbstractArray) at /Users/jakeireland/FaceDetection.jl/src/HaarLikeFeature.jl:97
Stacktrace:
 [1] (::Main.FaceDetection.Utils.var"#1#2"{Array{Float64,2}})(::Main.FaceDetection.Adaboost.HaarLikeFeature.HaarLikeObject) at ./none:0
 [2] iterate at ./generator.jl:47 [inlined]
 [3] collect(::Base.Generator{Array{Any,1},Main.FaceDetection.Utils.var"#1#2"{Array{Float64,2}}}) at ./array.jl:686
 [4] ensembleVote(::Array{Float64,2}, ::Array{Any,1}) at /Users/jakeireland/FaceDetection.jl/src/Utils.jl:87
 [5] #3 at /Users/jakeireland/FaceDetection.jl/src/Utils.jl:115 [inlined]
 [6] iterate at ./generator.jl:47 [inlined]
 [7] _collect(::Array{Array{Float64,2},1}, ::Base.Generator{Array{Array{Float64,2},1},Main.FaceDetection.Utils.var"#3#4"{Array{Any,1}}}, ::Base.EltypeUnknown, ::Base.HasShape{1}) at ./array.jl:699
 [8] collect_similar at ./array.jl:628 [inlined]
 [9] map at ./abstractarray.jl:2162 [inlined]
 [10] ensembleVoteAll(::Array{Array{Float64,2},1}, ::Array{Any,1}) at /Users/jakeireland/FaceDetection.jl/src/Utils.jl:115
 [11] main(::Bool, ::Bool) at /Users/jakeireland/FaceDetection.jl/src/FDA.jl:76
 [12] top-level scope at ./timing.jl:174
 [13] include(::Function, ::Module, ::String) at ./Base.jl:380
 [14] include(::Module, ::String) at ./Base.jl:368
 [15] exec_options(::Base.JLOptions) at ./client.jl:296
 [16] _start() at ./client.jl:506
in expression starting at /Users/jakeireland/FaceDetection.jl/src/FDA.jl:106

as of commit fa4352c

Fix face detection accuracy

After 456f8c6, face detection accuracy significantly decreased. Rectify this.