Similarity scores do not look meaningful

[huangyangyu]

Similarity scores do not look meaningful

[huangyangyu]

@ekarabulut Yes. In our test of LFW and YTF, all the images are aligned by mtcnn and util.align_box.
If you want to use evaluate.py script to test your own dataset, you should provide keypoints as parameter to util.align_box. You can get the keypoints by mtcnn or other landmark methods.

[huangyangyu]

I found the problem. The images you suppled to me are not aligned and cropped. Please refer to util.py
Moreover, you would better to normalize the feature vector. The effect may be better.

[ekarabulut]

@huangyangyu Thank you for your answer.

I understand each image should be aligned and cropped. I also assume this can be done by using functions given in util.py. Right?

I inspect the code (evaluate.py and featurer.py) and see that each image is passed through util.align_box function which presumably does the alignment job.

Therefore, I don't understand how further I can use util.py. It is already used in your code.

Do you refer to any other preprocesses other than things done in util.py? I'd be glad if you can clarify further.

EDIT: I checked the code one more time. This time, I saw the second parameter of util.align_box which is expected to have landmarks of the face features (eye_left, eye_right, nose etc) is always None. Therefore, util.align_box simply returns what is given (e.g. does nothing). I guess a python function that calculates landmarks of the detected face in the image is missing. Perhaps MTCNN is connected with the issue. It is possible I am getting it wrong. Any help is appreciated.

[ekarabulut]

Hi @huangyangyu

Thank you again for your feedback. I have recently had the opportunity to apply MTCNN (python implementation) to preprocess the images before giving to the SeqFace. It seems obtaining face landmark features and giving it to SeqFace works!

The new results with properly aligned face photos seem more meaningful.

• Faces of the same person give a cos similarity between -5 and -10
• Faces of different people give a cos similarity under -25

Therefore, I understand by applying a threshold, say -10, SeqFace is able to tell whether faces belong to different people or the same person. But I am not sure about how to pick a proper threshold for all cases as the value -10 is obtained as a result of my own experiments.

Can you tell me if my assumption is correct?

Thanks so far.

[isgursoy]

I think we can close this issue.