dedhiaparth98/face-recognition

Face Recognition Model trained with Siamese Network and Triplet Loss function in TensorFlow

Face Recognition Software

I have explained the details about the model architecture, dataset, and other implementational details in the blog post here. Here, I will provide the steps for using this repository.

Installations

The following steps are tested for ubuntu 20.04 with a python version of 3.8

sudo apt update
sudo apt upgrade
sudo snap install cmake

Clone the project to any directory and open the terminal in that directory. We will have to create some directories that will be essential for our storing images and features

mkdir data
mkdir features

I generally like to create a virtual environment for each of my projects so the next step is optional.

virtualenv -p /usr/bin/python3.8 venv
source venv/bin/activate

Now we will install all the pip packages required for running this applications.

pip install -r reqirement.txt 

If you want to just try out the Browser-Based UI tool or run the notebook, then you can download the pre-trained model weights from here.

Check your setup

After extracting the files, your directory should look like this.

.
├── app.py
├── config.py
├── data
├── features
├── logs
│   ├── func
│   ├── model
│   └── scalars
├── notebooks
│   ├── DatagGeneration.ipynb
│   ├── Real-time-prediction.ipynb
│   └── SiameseNetwork-TripletLoss.ipynb
├── README.md
├── reqirement.txt
├── siameseNetwork.py
├── static
│   ├── css
│   └── images
├── templates
│   ├── index.html
│   └── results.html
├── utils.py

Running the browser-based tool

If you have the same set of files and folders in the directory then you can run the following command

python app.py

The flask app will start and you will be able to collect training data for any new person and generate features for that person and check the real-time recognition. You can add as many people as you want to. The images collected from this tool will be added to the data folder and the corresponding features generated will be stored in the features folder.

Note : If you delete any person's images from the data folder, you need to delete the .pkl file inside the features folder as well. Also, the pickle file will be generated only when you hit submit images in the browser tool.

Running the Notebooks

You can start the jupyter-lab or jupyter notebook server and check the notebooks folder. Notebook Real-time-prediction.ipynb can be used for evaluating the model. It's the notebook version of the browser-based tool. However, the prediction in real-time webcam frame is much faster here as the browser sends API calls to the backend and each image frame of the video is send whereas here, it's not necessary.

Other instructions for running this notebook are provided in the notebook itself. However, the data directory is shared between this notebook and the browser-based tool.

Training from scratch

If you wish to train your model and get your own weights, then you can use SiameseNetwork-TripletLoss.ipynb. I had trained the same on colab and kept the lines of code for mounting the drive and other TensorFlow logging. Please refer to the blog post link above for learning more about the training details.

For Web Developers - I have a question

I have tried using socket connection as well as ajax calls for sending data to the backend while running prediction calls on the images. It was counter-intuitive to know that the socket connection was giving me a slower frame rate than the ajax calls.

The current implementation is with Ajax call but the commented code for the socket is kept in both frontend and backend. So if you know why is socket slow than ajax ? then please drop me a message on twitter or linkedin. Thanks in advance !!

References

1. O. M. Parkhi, A. Vedaldi, A. Zisserman, Deep Face Recognition, British Machine Vision Conference, 2015.
2. Q. Cao, L. Shen, W. Xie, O. M. Parkhi, A. Zisserman, VGGFace2: A dataset for recognising face across pose and age, International Conference on Automatic Face and Gesture Recognition, 2018.
3. F. Schroff, D. Kalenichenko, J. Philbin, FaceNet: A Unified Embedding for Face Recognition and Clustering, CVPR, 2015.
4. G. Koch, R. Zemel, R. Salakhutdinov, Siamese Neural Networks for One-shot Image Recognition, ICML deep learning workshop. Vol. 2. 2015.
5. https://github.com/rcmalli/keras-vggface
6. https://stanford.edu/~shervine/blog/keras-how-to-generate-data-on-the-fly
7. https://medium.com/datadriveninvestor/speed-up-your-image-training-on-google-colab-dc95ea1491cf

face-recognition's Issues

ValueError: Failed to find data adapter that can handle input: <class '__main__.DataGenerator'>, <class 'NoneType'>

hello, Parth Dedhia. Thanks for making the blog and this repo.

i'm working on my thesis, i need the Triplet Generator so that i come across to your blog and this repo, but i got some issues. can you help me, please?

My own custom function

def align_image(img):
    return alignment.align(96, img, alignment.getLargestFaceBoundingBox(img), 
                           landmarkIndices=AlignDlib.OUTER_EYES_AND_NOSE)

DataGenerator Class

class DataGenerator(tf.keras.utils.Sequence):
    def __init__(self, dataset_path, batch_size=32, shuffle=True):
        self.dataset = self.__curate_dataset(dataset_path)
        self.dataset_path = dataset_path
        self.shuffle = shuffle
        self.batch_size = batch_size
        self.no_of_people = len(list(self.dataset.keys()))
        self.on_epoch_end()
        
    def __getitem__(self, index):
        people = list(self.dataset.keys())[index * self.batch_size: (index + 1) * self.batch_size]
        P = []
        A = []
        N = []
        
        for person in people:
            anchor_index = random.randint(0, len(self.dataset[person])-1)
            a = self.__get_image(person, anchor_index)
            
            positive_index = random.randint(0, len(self.dataset[person])-1)
            while positive_index == anchor_index:
                positive_index = random.randint(0, len(self.dataset[person])-1)
            p = self.__get_image(person, positive_index)
            
            negative_person_index = random.randint(0, self.no_of_people - 1)
            negative_person = list(self.dataset.keys())[negative_person_index]
            while negative_person == person:
                negative_person_index = random.randint(0, self.no_of_people - 1)
                negative_person = list(self.dataset.keys())[negative_person_index]
            
            negative_index = random.randint(0, len(self.dataset[negative_person])-1)
            n = self.__get_image(negative_person, negative_index)
            P.append(p)
            A.append(a)
            N.append(n)
        A = np.asarray(A)
        N = np.asarray(N)
        P = np.asarray(P)
        return [A, P, N]
        
    def __len__(self):
        return self.no_of_people // self.batch_size
        
    def __curate_dataset(self, dataset_path):
        with open(os.path.join(dataset_path, 'list.txt'), 'r') as f:
            dataset = {}
            image_list = f.read().splitlines()
            for image in image_list:
                folder_name, file_name = image.split('/')
                if folder_name in dataset.keys():
                    dataset[folder_name].append(file_name)
                else:
                    dataset[folder_name] = [file_name]
        return dataset
    
    def on_epoch_end(self):
        if self.shuffle:
            keys = list(self.dataset.keys())
            random.shuffle(keys)
            dataset_ =  {}
            for key in keys:
                dataset_[key] = self.dataset[key]
            self.dataset = dataset_
            
    def __get_image(self, person, index):
        img = cv2.imread(os.path.join(self.dataset_path, os.path.join('images/' + person, self.dataset[person][index])))
        img = align_image(img) #my own custom function
        img = np.asarray(img, dtype=np.float64)
        img = preprocess_input(img)
        return img

data_generator = DataGenerator(dataset_path='./dataset/')

when i was passing the data_generator to model.fit_generator().

nn4_small2_train.compile(loss=None, optimizer='adam')
nn4_small2_train.fit_generator(data_generator, epochs=5, steps_per_epoch=20)

i got this error
ValueError: Failed to find data adapter that can handle input: <class '__main__.DataGenerator'>, <class 'NoneType'>

i don't want to loop the data generator like the way you did. i want to pass through the model.fit_generator() immediately

why is the output <class 'NoneType'>?

thanks

logs folder is missing

Provided pretrained VGG16 weights

Hey, excellent blog! Really well explained and written. I was trying to train a model myself as a side learning project, I am not able train the VGG16 model that you trained according to the paper link, but the link does not lead anywhere. Would you be kind enough to provide me the pretrianed weights that I can use to do transfer learning?