dataintelligencecrew/go-faiss

Go bindings for Faiss

go-faiss's Issues

cgo: inconsistent definitions for C.FaissMetricType

still this problem...
my test env has updated to go1.18.1 version. but still occur this problem.
I did not compiler faiss at test env. I just compiler faiss at my local Linux PC. and then move my local PC `s /usr/local/include/faiss /usr/lib /usr/lib64 /usr/local/share folder to my remote test env folder.
but it still occur cgo: inconsistent definitions for C.FaissMetricType. when I go get github.com/DataIntelligenceCrew/go-faiss.
somebody can help ?

type uint32 in assignment

after compile on
CentOS Linux release 7.9.2009 (Core)
x86_64

i got this :

github.com/DataIntelligenceCrew/go-faiss

/root/go/src/github.com/DataIntelligenceCrew/go-faiss/faiss.go:317:62: cannot use _Ctype_FaissMetricType(metric) (type _Ctype_FaissMetricType) as type uint32 in assignment
/root/go/src/github.com/DataIntelligenceCrew/go-faiss/faiss.go:338:81: cannot use _Ctype_FaissMetricType(metric) (type _Ctype_FaissMetricType) as type uint32 in assignment

See an illegal instruction error when I try to call the Train() API over an IVF100,SQ8 index built using IndexFactory(..)

I've built and installed FAISS per instructions on my ubuntu system. Has anyone seen anything of the likes or any pointers on how this could happen?

SIGILL: illegal instruction
--
PC=0x7fc8901c2cae m=8 sigcode=2
signal arrived during cgo execution
instruction bytes: 0xc5 0xfa 0x10 0x0 0x48 0x8b 0x85 0x60 0xff 0xff 0xff 0x8b 0x0 0x4c 0x8b 0x45goroutine 1122 [syscall]:
runtime.cgocall(0x124f1f0, 0xc000b55108)
/home/couchbase/.cbdepscache/exploded/x86_64/go-1.21.3/go/src/runtime/cgocall.go:157 +0x4b fp=0xc000b550e0 sp=0xc000b550a8 pc=0x40b70b
github.com/blevesearch/go-faiss._Cfunc_faiss_Index_train(0x7fc81ccdd400, 0x2ed2, 0xc00e2e0000)
_cgo_gotypes.go:513 +0x4b fp=0xc000b55108 sp=0xc000b550e0 pc=0x6e722b
github.com/blevesearch/go-faiss.(*faissIndex).Train.func1(0x7fc81ccdd400?, 0x2?, {0xc00e2e0000, 0x43dcbb?, 0x7fc81ccdd3e0?})
..

The compiler on the system -

gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
--
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Error in operation example: regex_error

I use faiss version 1.7.1, Error in operation example: regex_error 。
I tried to reduce the version, but the project didn't work。
I'm using version 1ea134eb83778310452111452b60f14391f92ad6 of faiss now，It's generally normal，However, idmap is invalid, and add_WITH_IDS cannot be used： “Error in virtual void faiss::Index::add_with_ids(faiss::Index::idx_t, const float*, const idx_t*) at /opt/faiss/faiss/Index.cpp:39: add_with_ids not implemented for this type of index”

help: Can help to achieve the FaissIndexIVFFlat

i tried,but i don't know c++ well.
can you add the FaissIndexIVFFlat。
thanks

make -C build Linking CXX executable faiss_test error

Hello, I encountered the following problems, I do not know how to solve, could you help me to take a look？

[ 96%] Building CXX object tests/CMakeFiles/faiss_test.dir/test_pq_encoding.cpp.o
[ 96%] Building CXX object tests/CMakeFiles/faiss_test.dir/test_sliding_ivf.cpp.o
[ 98%] Building CXX object tests/CMakeFiles/faiss_test.dir/test_threaded_index.cpp.o
[ 98%] Building CXX object tests/CMakeFiles/faiss_test.dir/test_transfer_invlists.cpp.o
[100%] Linking CXX executable faiss_test
/mnt/c/GOPATH/src/github.com/facebookresearch/faiss/build/tests/faiss_test: symbol lookup error: /mnt/c/GOPATH/src/github.com/facebookresearch/faiss/build/tests/faiss_test: undefined symbol: _ZNK5faiss5Index6assignElPKfPll
CMake Error at /usr/share/cmake/Modules/GoogleTestAddTests.cmake:77 (message):
Error running test executable.

Path: '/mnt/c/GOPATH/src/github.com/facebookresearch/faiss/build/tests/faiss_test'
Result: 127
Output:

Call Stack (most recent call first):
/usr/share/cmake/Modules/GoogleTestAddTests.cmake:173 (gtest_discover_tests_impl)

make[2]: *** [tests/CMakeFiles/faiss_test.dir/build.make:311: tests/faiss_test] Error 1
make[2]: *** Deleting file 'tests/faiss_test'
make[2]: Leaving directory '/mnt/c/GOPATH/src/github.com/facebookresearch/faiss/build'
make[1]: *** [CMakeFiles/Makefile2:1520: tests/CMakeFiles/faiss_test.dir/all] Error 2
make[1]: Leaving directory '/mnt/c/GOPATH/src/github.com/facebookresearch/faiss/build'
make: *** [Makefile:146: all] Error 2
make: Leaving directory '/mnt/c/GOPATH/src/github.com/facebookresearch/faiss/build'

Building a Windows exe for distribution

Hello,

We have an application built as a Windows executable that is being distributed to customers. Do we need to bundle the faiss C++ library / header files (as DLL perhaps?) if we use go-faiss? How to do this?

Thank you.

regex_error , std::exception

when I call faiss.IndexFactory it occur. can anybody help? i dont know what the problem it is.

Undefined symbols for architecture x86_64

github.com/DataIntelligenceCrew/go-faiss

ld: warning: ignoring file /usr/local/lib/libfaiss_c.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
Undefined symbols for architecture x86_64:
"_faiss_IDSelectorBatch_new", referenced from:
__cgo_b3ad317e889f_Cfunc_faiss_IDSelectorBatch_new in _x007.o
(maybe you meant: __cgo_b3ad317e889f_Cfunc_faiss_IDSelectorBatch_new)
"_faiss_IDSelectorRange_new", referenced from:
。。。。。

it happen at macbook M1 when I go build。 how should I cmake faiss for macOS-arm64 ,aka macbook M1

only use 1 cpu

build from centos7 ; but only use 1 cpu

cmake -B build -DFAISS_ENABLE_GPU=OFF -DFAISS_ENABLE_C_API=ON -DBUILD_SHARED_LIBS=ON -DFAISS_ENABLE_PYTHON=ON -DFAISS_OPT_LEVEL=avx2 -DCMAKE_BUILD_TYPE=Release -DBLA_VENDOR=Intel10_64_dyn -DPython_EXECUTABLE=/usr/bin/python3 .

can i use static library and use GPU to calculate vector？

no such file or directory

：../../../../../../../../github.com/DataIntelligenceCrew/go-faiss/autotune.go:5:36: error: faiss/c_api/AutoTune_c.h: No such file or directory

anyone meet this problem? please help.

i already mv /faiss/build/c_api/libfaiss_c.so to /usr/lib/libfaiss_c.so and /usr/local/lib/libfaiss_c.so

but when i go build my go program . error occur.

Fail to use this library

Hi,

I am trying to run the example, but when I run go get github.com/DataIntelligenceCrew/go-faiss it shows:

# github.com/DataIntelligenceCrew/go-faiss
../../autotune.go:12:6: could not determine kind of name for C.FaissParameterSpace
../../autotune.go:39:2: could not determine kind of name for C.faiss_ParameterSpace_free
../../autotune.go:18:10: could not determine kind of name for C.faiss_ParameterSpace_new
../../autotune.go:29:7: could not determine kind of name for C.faiss_ParameterSpace_set_index_parameter
../../autotune.go:27:8: could not determine kind of name for C.free

I use local Mac env, and I have copy libfaiss_c.so to /usr/local/lib/

By any chance you know the reason? Thanks!

Just tried again, the error becomes:

# github.com/DataIntelligenceCrew/go-faiss
./autotune.go:5:10: fatal error: 'faiss/c_api/AutoTune_c.h' file not found
#include <faiss/c_api/AutoTune_c.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Linking CXX shared library libfaiss_python_callbacks.dylib。Undefined symbols for architecture arm64

[ 61%] Linking CXX shared library libfaiss_python_callbacks.dylib
Undefined symbols for architecture arm64:
"_PyBytes_AsString", referenced from:
PyCallbackIOReader::operator()(void*, unsigned long, unsigned long) in python_callbacks.cpp.o
"_PyBytes_FromStringAndSize", referenced from:
PyCallbackIOWriter::operator()(void const*, unsigned long, unsigned long) in python_callbacks.cpp.o
"_PyBytes_Size", referenced from:
PyCallbackIOReader::operator()(void*, unsigned long, unsigned long) in python_callbacks.cpp.o
"_PyGILState_Ensure", referenced from:
(anonymous namespace)::PyThreadLock::PyThreadLock() in python_callbacks.cpp.o
"_PyGILState_Release", referenced from:
(anonymous namespace)::PyThreadLock::~PyThreadLock() in python_callbacks.cpp.o
"_PyObject_CallFunction", referenced from:
PyCallbackIOWriter::operator()(void const*, unsigned long, unsigned long) in python_callbacks.cpp.o
PyCallbackIOReader::operator()(void*, unsigned long, unsigned long) in python_callbacks.cpp.o
"__Py_Dealloc", referenced from:
_Py_DECREF(_object*) in python_callbacks.cpp.o
"faiss::FaissException::FaissException(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, char const*, char const*, int)", referenced from:
PyCallbackIOWriter::operator()(void const*, unsigned long, unsigned long) in python_callbacks.cpp.o
PyCallbackIOReader::operator()(void*, unsigned long, unsigned long) in python_callbacks.cpp.o
"faiss::IOReader::fileno()", referenced from:
vtable for PyCallbackIOReader in python_callbacks.cpp.o
"faiss::IOWriter::fileno()", referenced from:
vtable for PyCallbackIOWriter in python_callbacks.cpp.o
"typeinfo for faiss::FaissException", referenced from:
PyCallbackIOWriter::operator()(void const*, unsigned long, unsigned long) in python_callbacks.cpp.o
PyCallbackIOReader::operator()(void*, unsigned long, unsigned long) in python_callbacks.cpp.o
"typeinfo for faiss::IOReader", referenced from:
typeinfo for PyCallbackIOReader in python_callbacks.cpp.o
"typeinfo for faiss::IOWriter", referenced from:
typeinfo for PyCallbackIOWriter in python_callbacks.cpp.o
"vtable for faiss::FaissException", referenced from:
faiss::FaissException::~FaissException() in python_callbacks.cpp.o
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
"vtable for faiss::IOReader", referenced from:
faiss::IOReader::IOReader() in python_callbacks.cpp.o
faiss::IOReader::~IOReader() in python_callbacks.cpp.o
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
"vtable for faiss::IOWriter", referenced from:
faiss::IOWriter::IOWriter() in python_callbacks.cpp.o
faiss::IOWriter::~IOWriter() in python_callbacks.cpp.o
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [faiss/python/libfaiss_python_callbacks.dylib] Error 1
make[1]: *** [faiss/python/CMakeFiles/faiss_python_callbacks.dir/all] Error 2
make: *** [all] Error 2

Did anyone meet this problem? need some help. where is libfaiss_python_callbacks.dylib from. how should I fix it.

can‘t find faiss/c_api/IndexFlat_c.h

after build ，don't have _c.h file

-- Up-to-date: /usr/local/include/faiss/IVFlib.h

github.com/DataIntelligenceCrew/go-faiss

/root/go_projects/src/github.com/DataIntelligenceCrew/go-faiss/faiss.go:11:37: fatal error: faiss/c_api/IndexFlat_c.h: No such file or directory
#include <faiss/c_api/IndexFlat_c.h>
^
compilation terminated.

please help me。

./test: symbol lookup error: /usr/lib/libfaiss_c.so: undefined symbol: __cxa_init_primary_exception

I copy the libfaiss_c.so to my test env server. when I execute my go program . It occur this error.
by the way. the libfaiss_c.so file is not compile at my test env server. it compile at local linux PC.
what does the error mean?

Not compilable in Mac OS

Hi, the setup commands in the wiki works fine with me in Ubuntu OS, but i got some error when setup on Mac. Just want to check if it can be compiled on Mac?

make -C build
[ 61%] Built target faiss
[ 62%] Built target swigfaiss_swig_compilation
[ 62%] Linking CXX shared library libfaiss_python_callbacks.dylib
Undefined symbols for architecture x86_64:
"_PyBytes_AsString", referenced from:
PyCallbackIOReader::operator()(void*, unsigned long, unsigned long) in python_callbacks.cpp.o
"_PyBytes_FromStringAndSize", referenced from:
PyCallbackIOWriter::operator()(void const*, unsigned long, unsigned long) in python_callbacks.cpp.o
"_PyBytes_Size", referenced from:
PyCallbackIOReader::operator()(void*, unsigned long, unsigned long) in python_callbacks.cpp.o
"_PyGILState_Ensure", referenced from:
(anonymous namespace)::PyThreadLock::PyThreadLock() in python_callbacks.cpp.o
"_PyGILState_Release", referenced from:
(anonymous namespace)::PyThreadLock::~PyThreadLock() in python_callbacks.cpp.o
"_PyObject_CallFunction", referenced from:
PyCallbackIOWriter::operator()(void const*, unsigned long, unsigned long) in python_callbacks.cpp.o
PyCallbackIOReader::operator()(void*, unsigned long, unsigned long) in python_callbacks.cpp.o
"faiss::FaissException::FaissException(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, char const*, char const*, int)", referenced from:
PyCallbackIOWriter::operator()(void const*, unsigned long, unsigned long) in python_callbacks.cpp.o
PyCallbackIOReader::operator()(void*, unsigned long, unsigned long) in python_callbacks.cpp.o
"faiss::IOReader::fileno()", referenced from:
vtable for PyCallbackIOReader in python_callbacks.cpp.o
"faiss::IOWriter::fileno()", referenced from:
vtable for PyCallbackIOWriter in python_callbacks.cpp.o
"typeinfo for faiss::FaissException", referenced from:
PyCallbackIOWriter::operator()(void const*, unsigned long, unsigned long) in python_callbacks.cpp.o
PyCallbackIOReader::operator()(void*, unsigned long, unsigned long) in python_callbacks.cpp.o
"typeinfo for faiss::IOReader", referenced from:
typeinfo for PyCallbackIOReader in python_callbacks.cpp.o
"typeinfo for faiss::IOWriter", referenced from:
typeinfo for PyCallbackIOWriter in python_callbacks.cpp.o
"vtable for faiss::FaissException", referenced from:
faiss::FaissException::~FaissException() in python_callbacks.cpp.o
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
"vtable for faiss::IOReader", referenced from:
faiss::IOReader::IOReader() in python_callbacks.cpp.o
faiss::IOReader::~IOReader() in python_callbacks.cpp.o
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
"vtable for faiss::IOWriter", referenced from:
faiss::IOWriter::IOWriter() in python_callbacks.cpp.o
faiss::IOWriter::~IOWriter() in python_callbacks.cpp.o
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [faiss/python/libfaiss_python_callbacks.dylib] Error 1
make[1]: *** [faiss/python/CMakeFiles/faiss_python_callbacks.dir/all] Error 2
make: *** [all] Error 2

cgo: inconsistent definitions for C.FaissMetricType

when I go get github.com/DataIntelligenceCrew/go-faiss . it occur.
cgo: inconsistent definitions for C.FaissMetricType.

and I have move libfaiss_c.so to /usr/lib

anyone meet this problem?