habrman/facerecognition

Webcam face recognition using tensorflow and opencv

FaceRecognition

Webcam face recognition using tensorflow and opencv. The application tries to find faces in the webcam image and match them against images in an id folder using deep neural networks.

Dependencies

• OpenCv
• Tensorflow
• Scikit-learn
• easygui

Inspiration

Models, training code and inspriation can be found in the facenet repository. Multi-task Cascaded Convolutional Networks are used for facial and landmark detection while an Inception Resnet is used for ID classification. A direct link to the pretrained Inception Resnet model can be found here.

How to

Get the model from facenet and setup your id folder. The id folder should contain subfolders, each containing at least one image of one person. The subfolders should be named after the person in the folder since this name is used as output when a match is found.

E.g. id folder named ids containing subfolders Adam and Eve, each containing images of the respective person.

├── ids
│   ├── Adam
│   │   ├── Adam0.png
│   │   ├── Adam1.png
│   ├── Eve
│   │   ├── Eve0.png

Download and unpack the model to a folder and run python3 main.py ./folder/model.pb ./ids/ to start the program. Make sure to replace ./folder/model.pb with the path to the downloaded model.

Visualization hotkeys:

• l - toggle facial landmarks
• b - toggle bounding box
• i - toggle id
• f - toggle frames per second
• s - save image face detections to id folder

facerecognition's Issues

Remove padding

Is there a way to remove face padding and get the exact bounding box fit to face?
Thank you

Error Running Code

I tried to run the code but despite installing sklearn with anaconda prompt ,it says 'module sklearn not found'.

how to train pic to match face id and use mtcnn which features to detect

Recognize a person's profile

Hello author, when I was running the program, I found that the program could not detect my face when I turned my head over 90°, and also when I put on a mask. May I ask if there is a solution to this problem? Thanks.

No module name CV2

My program is giving error no module found naming cv2 in line 5

How do add new person image in model runtime.

Ok so here is the scenario:

1. My model detects unknown person image, and captures it face, generate an unique id and save it in ids folder.
2. My model again detects that face and returns it ID all in run time.

Right now, when I start the model it gets all the images and train itself, but I do not wana start my model everytime I add new image. Is there any way I can do this in real time ?

Add video source input

Make it possible to run on a video input instead of webcam

Doesn't run on OSX Mojave 10.14.2

Folder setup:
└── FaceRecognition
├── 20170512-110547
│ ├── 20170512-110547.pb
│ ├── model-20170512-110547.ckpt-250000.data-00000-of-00001
│ ├── model-20170512-110547.ckpt-250000.index
│ └── model-20170512-110547.meta
├── LICENSE
├── README.md
├── pycache
│ └── detect_and_align.cpython-37.pyc
├── det1.npy
├── det2.npy
├── det3.npy
├── detect_and_align.py
├── example.png
├── ids
│ ├── Ben
│ │ ├── Ben0.png
│ │ └── Ben1.jpg
│ └── James
│ └── James0.jpg
└── main.py

Error

bunker$ python3 main.py ./20170512-110547/20170512-110547.pb ./ids/
ARGS: Namespace(id_folder=['./ids/'], model='./20170512-110547/20170512-110547.pb', threshold=1.2)
Loading model filename: ./20170512-110547/20170512-110547.pb
graph
Traceback (most recent call last):
File "main.py", line 188, in
main(parser.parse_args())
File "main.py", line 102, in main
load_model(args.model)
File "main.py", line 90, in load_model
tf.import_graph_def(graph_def, name='')
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 258, in import_graph_def
op_def = op_dict[node.op]
KeyError: 'FIFOQueueV2'

Memory lack! with Tensorflow oom on GPU

@habrman !
First, thank you very much for your nice project and the guidance, everything work well for me.

Then, I faced a problem with huge number of subjects due to the lack of memory. My GPU supports 8 GB memory but I faced error "GPU Memory Allocation" when I use 2000+ subjects of LFW.
I found that I have to decrease the batch_size of the data in the network to use smaller parts of memory allocations but I am not sure where you learn.

here is the line that I received error
self.embeddings = sess.run(embeddings, feed_dict=feed_dict)
Could you please help me to solve it?
Thank you in advance

Can we use for large scale Video Surveillance?

Low FPS

I'm getting around 5 to 6 FPS. How to increase FPS.

Dataset used to train mtcnn

Hello author.I want to ask what kind of dataset are used when you are training the mtcnn model ? can you share your training code if it is convenient.

Wrong Recognition of Faces

Hi Habrman,

I am testing this algorithm for my research purpose for the past 5 months, sometimes i see wrong faces are recognised(Example: Face ID: A is Recognised as Face ID:B) or Similar Bald Persons are wrongly recognised.

I have increased the threshold to threshold = [0.8, 0.7, 0.7], inspite of this, I do get false recognition's, Can you please suggest me few other ways to increase the detection rate

Thanks
Guru

Save and load embeddings

Hi @habrman and thanks for sharing your code.
As my database includes a number of images I want to get rid of making embeddings every time.
I tested saving and loading embeddings using this syntax:

#for saving:
np.save('embeddings.npy' , self.embeddings)
#for loading
self.embeddings = np.load('embeddings.npy')

in the IdData Class.
but it leads to error:

IndexError: list index out of range

"main.py" execute error

Hello.
I got some problem when execute your codes.

me~~~$ python3 main.py ./20170512-110547/ ./pics/
Couldn't import dot_parser, loading of dot files will not be possible.
2018-01-23 16:37:29.142315: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-01-23 16:37:29.541278: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.62
pciBusID: 0000:02:00.0
totalMemory: 10.91GiB freeMemory: 10.45GiB
2018-01-23 16:37:29.880962: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 1 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.62
pciBusID: 0000:03:00.0
totalMemory: 10.91GiB freeMemory: 10.32GiB
2018-01-23 16:37:30.222775: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 2 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.721
pciBusID: 0000:81:00.0
totalMemory: 10.91GiB freeMemory: 10.32GiB
2018-01-23 16:37:30.222992: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Device peer to peer matrix
2018-01-23 16:37:30.223055: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1051] DMA: 0 1 2
2018-01-23 16:37:30.223067: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1061] 0:   Y Y N
2018-01-23 16:37:30.223075: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1061] 1:   Y Y N
2018-01-23 16:37:30.223083: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1061] 2:   N N Y
2018-01-23 16:37:30.223103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0, compute capability: 6.1)
2018-01-23 16:37:30.223113: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:1) -> (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:03:00.0, compute capability: 6.1)
2018-01-23 16:37:30.223122: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:2) -> (device: 2, name: GeForce GTX 1080 Ti, pci bus id: 0000:81:00.0, compute capability: 6.1)
Model directory: ./20170512-110547/
Metagraph file: model-20170512-110547.meta
Checkpoint file: model-20170512-110547.ckpt-250000
WARNING:tensorflow:The saved meta_graph is possibly from an older release:
'model_variables' collection should be of type 'byte_list', but instead is of type 'node_list'.
/usr/local/lib/python3.5/dist-packages/scipy/misc/pilutil.py:479: FutureWarning: Conversion of the second argument of issubdtype from `int` to `np.signedinteger` is deprecated. In future, it will be treated as `np.int64 == np.dtype(int).type`.
  if issubdtype(ts, int):
/usr/local/lib/python3.5/dist-packages/scipy/misc/pilutil.py:482: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  elif issubdtype(type(size), float):
Traceback (most recent call last):
  File "main.py", line 227, in <module>
    main(parse_arguments(sys.argv[1:]))
  File "main.py", line 141, in main
    id_dataset = id_data.get_id_data(args.id_folder[0], pnet, rnet, onet, sess, embeddings, images_placeholder, phase_train_placeholder)
  File "/u/data/arith/face/FaceRecognition/id_data.py", line 31, in get_id_data
    id_dataset[i].embedding = emb[i, :]
IndexError: index 5 is out of bounds for axis 0 with size 5

I didn't solve this problem.
Could you help me?

How to load model and images in this project

Please author help me to run this program, I'm trying to learn something from your work but somehow I cannot able run this program. it gives me this error

"usage: main.py [-h] [-t THRESHOLD] model id_folder [id_folder ...]
main.py: error: the following arguments are required: model, id_folder"

please anybody help me :(

What algorithm did you use in this project?

Hello Sir, I would like to know algorithm that you used. if possible, I want to know about working process from Start to End. Thank you.

why the output name is ids folder?

hi!
I have a problem with showing the id name of the recognized faces. when the face is detected, the output name is the id folder name instead of the person id.
my ids folder path is as follows: ids/x/x0.png
but what I get in the console is: "Hi ids! Distance: 0.58" and the same name appears under the identified face.

thanks,

Name not showing

I ran the code,it recognizes the face but instead of displaying name of one of sub folders after identifying,it writes 'face recognition-master' instead of it.

Where does code be used to recognition in file detect_facese_real_time.py ?

And Where does code be used Facenet algorithm to recognition in file detect_facese_real_time.py ? Does you use file detect_facese_real_time.py only detect ?
Thank you !!!

How to add UNknown label for untrained images

HI,

I am using this code to identify unknown faces that are not trained, so Can you please let me know, where to add "Unknown" label, so that I can take screen shot of those unknown persons to train later.

Thanks
Guru