zjulearning/efanna_graph

an Extremely Fast Approximate Nearest Neighbor graph construction Algorithm framework

efanna_graph's Issues

error: double free or corruption (!prev)

when I try to execute this command on my own dataset
double free or corruption (!prev) occured

error: ‘idx_t’ is not a member of ‘faiss::Index’

Recall stays 0.0001

I am trying to generate a graph using efanna_graph. I have a dataset of 4000 images. I have calculated and wrote all SIFT descriptors to a .fvecs file and used that to generate the graph. Unfortunately efanna_graph recall never went above 0.0001. I believe it is an issue with the way I write descriptors to .fvecs.

I have tried to write .fvecs multiple ways. The code I am using now is this: write_to_fvecs . As you can see after I have calculated descriptors for each image and concatenated these descriptors in a single array I write them to .fvecs using:
vectorArray.astype(np.int32).tofile('./my_sift_descriptors.fvecs')
As you can see I use np.int32 which seems wrong to me. The reason for using np.int32 is as follows.

First I tried writing to file like this:
vectorArray.astype().tofile('./vanbeeklederwaren_astype_int32.fvecs')
But when I start efanna_graph test_nndescent I get this message: "data dimension: 1124073472
Floating point exception (core dumped)".

Then I tried writing to file like this(which seems to me is the correct way to this):
vectorArray.astype(np.float32).tofile('./vanbeeklederwaren_astype_int32.fvecs')
But again when running efanna_graph I get this message: "data dimension: 1124073472
Floating point exception (core dumped)".

Then I used this snippet: read_fvecs which you can use to read fvecs files in python. I used this snippet to read the first 4 bytes of 4 different files. The first:
The fvecs file provided by TexMex showed the first 4 bytes to be of type of float32 and the value was 1.8e-43.

The file saved without specifying a type was also of type float32 but displayed 128.0 when printed.

The file saved as float32 also was of type float32 and also displayed 128.0 when printed.

The file saved as int32 also was of type but displayed 1.8e-43 when printed.

I assumed the last file should be correct, thus I continued and calculated all my descriptors, saved them to .fvecs and started efanna_graph. However the training did no go as expected and the recall never went above 0.0001. The parameters I used: 200 200 20 10 100.

I can't seem to find a solution. Can you please provide your snippet on how you compute SIFT descriptors and save these to .fvecs file?

Thank you.

Can't compile: ‘index_factory’ is not a member of ‘faiss’

Hello, i'm trying to compile with faiss.
But after command "make" there is error:

/home/kimal/Kema/gitty/efanna_graph/src/index_pq.cpp: In member function ‘virtual void efanna2e::IndexPQ::Build(size_t, const float*, const efanna2e::Parameters&)’:
/home/kimal/Kema/gitty/efanna_graph/src/index_pq.cpp:46:18: error: ‘index_factory’ is not a member of ‘faiss’
   46 |   index = faiss::index_factory(dimension_, pq_index_key.c_str());
      |   

I'm using Fedora.
I've already compiled faiss in "extern_libraries" directory.
Is there something I've missed?

Thank you for reading. I'll be glad to see any response.

what is GraphAdd used for? how to use it?

I find a function GraphAdd in index_graph.cpp. Is it used for retrain the graph when we add data? And how can I use it?