Giter Site home page Giter Site logo

Comments (2)

RioMichael avatar RioMichael commented on August 23, 2024

I also got the duplicate problem (but didn't get any search time problem). The duplicate problem occurred even when I tried to search with about 1000-NN (tested both BKT and KDT with data with about 100 dimension, and size of around 100,000 using Cosine as distance method), and it happened not only when using Python wrapper, but also when directly using IndexSearcher (so I guess the problem lies withing the source code).
I'm currently reading the source code, but since I'm not very familiar with c++, I haven't found any problem yet. Anyone has some ideas or solutions on this?

My environment:

Python 2.7
Windows 10/Ubuntu 16.04

from sptag.

RioMichael avatar RioMichael commented on August 23, 2024

So I was focusing on testing different parameters in the past two weeks, and didn't look too deep into the source code. But I do temporally solve the duplicate problem by adding a check when adding a node to the result query (QueryResultSet.h under SPTAGLib):

bool AddPoint(const int index, float dist)
    {
	for (int i = 0; i < m_results.Length(); i++) {
		if (index == m_results[i].VID) {
			return false;
		}
	}
        if (dist < m_results[0].Dist || (dist == m_results[0].Dist && index < m_results[0].VID))
        {
            m_results[0].VID = index;
            m_results[0].Dist = dist;
            Heapify(m_resultNum);
            return true;
        }
        return false;
    }

Although making this change can avoid the duplicates, I believe that this is not the alternative solution as the same node should not be able to reach AddPoint function more than once (maybe something wrong when checking the hash table, haven't looked at the hash table yet).
@Arctanxy Have you got a better solution? Also, did you get good result on your face dataset? I'm also testing with a face database (256 dimension, and using L2 distance), but did not get good result (have tried to tuning almost all the parameters.) But do get good results on all the datasets on https://github.com/erikbern/ann-benchmarks

from sptag.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.