Building of a binary hash code image similarity search application

[jolibrain]

Building of a binary hash code image similarity search application

[beniz]

Given that there's now a demo provided for image similarity search, I am closing this issue. Any further development may reopen it.

[beniz]

Good initiative! Link to the paper: http://www.iis.sinica.edu.tw/~kevinlin311.tw/cvprw15.pdf

Original implementation, using an old version of Caffe: https://github.com/kevinlin311tw/caffe-cvprw15

I believe this can be achieved in a few steps:

• Setup the appropriate .prototxt file with added latent layer, see https://gist.github.com/kevinlin311tw/266d4150a1db5810398e for replicate of the original on CIFAR
• Getting access to the net's inner layers through the API, and this is already on our todolist in a dedicated ticket, see #56
• Implement thresholding into dd for creating the binary features out of any layer. I believe that it makes sense to have this right into deepdetect, typically with a boolean binarized passed to the output connector
• Figure out what proper format is needed for indexing into ES: Hamming distance is a comparison metric of choise, I am not sure yet how ES can be configured to appropriately compute the nearest neighbors based on Hamming (or any other non-textual) distance
• Optionally, create a simple Web interface for looking at the results. An example to start from lies in the demo/imgdetect repository

Note that some other works like visual search from Pinterest do not rely on an additional latent layer but use binarized features from the existing net's layers directly. This means that an intermediary step for testing the whole pipeline does not forcely require the additional latent layer.

I understand that getting access to Caffe's inner layer is a blocker here. Depending on your C++ skills you can dig into the caffelib::predict function and gather preliminary thoughts and/or code, or let that to me, but I won't be able to work on it till after the week-end.

[beniz]

@masakinakada PR #59 implements most of #56 and you should now be able to move forward. Visual search should not be far. Please let me know of any progress and difficulties, if any.

[masakinakada]

Hi Em,

Thank you so much for this update. You are my life saver. I was trying to modify other part of the code but was a bit struggling.

I will try to complete it with your awesome update!
Thank you very much. I will give you update on my progress.

Masaki

Sent from iPhone

On Feb 3, 28 Heisei, at 11:14 PM, Emmanuel Benazera [email protected] wrote:

@masakinakada PR #59 implements most of #56 and you should now be able to move forward. Visual search should not be far. Please let me know of any progress and difficulties, if any.

―
Reply to this email directly or view it on GitHub.

[beniz]

PR #73 gives a full example of both indexing and searching of similar images with good performances.

[masakinakada]

Awesome! Thanks a lot Ema!

On Fri, Feb 26, 2016 at 11:38 PM, Emmanuel Benazera <
[email protected]> wrote:

PR #73 https://github.com/beniz/deepdetect/pull/73 gives a full example
of both indexing and searching of similar images with good performances.

—
Reply to this email directly or view it on GitHub
https://github.com/beniz/deepdetect/issues/55#issuecomment-189600748.