avilash/pytorch-siamese-triplet

One-Shot Learning with Triplet CNNs in Pytorch

Deep metric learning using Triplet network in PyTorch

The following repository contains code for training Triplet Network in Pytorch
Siamese and Triplet networks make use of a similarity metric with the aim of bringing similar images closer in the embedding space while separating non similar ones.
Popular uses of such networks being -

• Face Verification / Classification
• Learning deep embeddings for other tasks like classification / detection / segmentation

Paper - Deep metric learning using Triplet network

Installation

Install PyTorch

pip install -r requirements.txt  

Demo

Colab notebook with pretrained weights

Training

python train.py --cuda  

This by default will train on the MNIST dataset

MNIST / FashionMNIST

python train.py --result_dir results --exp_name MNIST_exp1 --cuda --dataset <manist>/<fmnist>  

To create a tSNE visualisation

python tsne.py --ckp <path to model>  

The embeddings and the labels are stored in the experiment folder as a pickle file, and you do not have to run the model everytime you create a visualisation. Just pass the saved embeddings as the --pkl parameter

python tsne.py --pkl <path to stored embeddings>  

Sample tSNE visualisation on MNIST

VGGFace2

Specify the location of the dataset in test.yaml
The directory should have the following structure

+-- root
|   +-- train
|       +-- class1
|           +-- img1.jpg
|           +-- img2.jpg
|           +-- img3.jpg
|       +-- class2
|       +-- class3
|   +-- test
|       +-- class4
|       +-- class5

python train.py --result_dir results --exp_name VGGFace2_exp1 --cuda --epochs 50 --ckp_freq 5 --dataset vggface2 --num_train_samples 32000 --num_test_samples 5000 --train_log_step 50 

Custom Dataset

Specify the location of the dataset in test.yaml
The directory should have the following structure

+-- root
|   +-- train
|       +-- class1
|           +-- img1.jpg
|           +-- img2.jpg
|           +-- img3.jpg
|       +-- class2
|       +-- class3
|   +-- test
|       +-- class4
|       +-- class5

python train.py --result_dir results --exp_name Custom_exp1 --cuda --epochs 50 --ckp_freq 5 --dataset custom --num_train_samples 32000 --num_test_samples 5000 --train_log_step 50 

TODO

• Train on MNIST / FashionMNIST
• Train on a public dataset
• Multi GPU Training
• Custom Dataset
• Include popular models - ResneXT / Resnet / VGG / Inception

pytorch-siamese-triplet's Issues

opencv BGR bug

I've been browsing the code, and it seems that you use opencv.imread() to read the images, which produces BGR output. But you still use the normalisation parameters for rgb: mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]. You can change BGR to RGB using the following: cv2.cvtColor(img, cv2.COLOR_BGR2RGB). Thanks for your work!

In PyTorch's examples, they read with PIL, which reads in RGB by default, but they use the same constants.
Reference: https://pytorch.org/hub/pytorch_vision_alexnet/

How to make tsne custom dataset

I want to make tsne dataset for a custom dataset

Custom dataset

Do you have any code like mnist.getTriplet for custom data?
Thanks!

choosing architectures

Hi, How I can use this code for my own images dataset

How to Perform Inference

Hi,

How can I perform inference from the model on an image or a video? Thanks

How to do inference on an image?

Hi,your work is helpful to me.
I have a question about how to do inference on an image?I couldn't find the relevant code.The tsne.py couldn't load the ckpt.

adding tensorboard

the current codebase does not have tensorboard functionalities. it would be great to add that