Errors when retrieval back

[Maxwells-Demons]

Errors when retrieval back

[rom1504]

You need to specify you use L/14 model when starting clip back

…

On Fri, Mar 1, 2024, 4:33 PM Maxwells_Ayakashi ***@***.***> wrote: First, thanks of your fruitful work! Following the guide of here <https://github.com/rom1504/laion-prepro/blob/main/laion400m/embeddings-indices-hosting/README.md>. I have downloaded laion400 datasets, and caculated its embedding and indices. Then I use command below to start the retrieval server: clip-retrieval back --port 1234 --indices-paths indices_paths.json --enable_hdf5 True --enableaiss_memory_mapping True --use_jit False --clip_model ViT-L/14 --use_arrow True The content of indices_path.json is: {"laion_400m": "/public_bme/data/xiedian/laion400m/index_folder"} And when I use such query: QQ.20240301232643.png (view on web) <https://github.com/rom1504/clip-retrieval/assets/54426652/9966ff61-5191-4771-8fbf-f0c924d2d9b7> The terminal reported following stacktrace: Traceback (most recent call last): File "/home_data/home/xiedian2023/.conda/envs/mindeye/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request rv = self.dispatch_request() File "/home_data/home/xiedian2023/.conda/envs/mindeye/lib/python3.10/site-packages/flask/app.py", line 1469, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) File "/home_data/home/xiedian2023/.conda/envs/mindeye/lib/python3.10/site-packages/flask_restful/__init__.py", line 489, in wrapper resp = resource(*args, **kwargs) File "/home_data/home/xiedian2023/.conda/envs/mindeye/lib/python3.10/site-packages/flask/views.py", line 109, in view return current_app.ensure_sync(self.dispatch_request)(**kwargs) File "/home_data/home/xiedian2023/.conda/envs/mindeye/lib/python3.10/site-packages/flask_restful/__init__.py", line 604, in dispatch_request resp = meth(*args, **kwargs) File "<decorator-gen-2>", line 2, in post File "/home_data/home/xiedian2023/.conda/envs/mindeye/lib/python3.10/site-packages/prometheus_client/context_managers.py", line 80, in wrapped return func(*args, **kwargs) File "/home_data/home/xiedian2023/.conda/envs/mindeye/lib/python3.10/site-packages/clip_retrieval/clip_back.py", line 492, in post return self.query( File "/home_data/home/xiedian2023/.conda/envs/mindeye/lib/python3.10/site-packages/clip_retrieval/clip_back.py", line 455, in query distances, indices = self.knn_search( File "/home_data/home/xiedian2023/.conda/envs/mindeye/lib/python3.10/site-packages/clip_retrieval/clip_back.py", line 362, in knn_search distances, indices, embeddings = index.search_and_reconstruct(query, num_result_ids) File "/home_data/home/xiedian2023/.conda/envs/mindeye/lib/python3.10/site-packages/faiss/class_wrappers.py", line 378, in replacement_search_and_reconstruct assert d == self.d AssertionError BTW, someone engaged this problem before: #212 <#212>. So I use --use_arrow True and replace parquet files in laion400m/index_folder/metadata with generated 0.arrow. But the problem still. So is there something wrong in my settings? — Reply to this email directly, view it on GitHub <#366>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAR437T6FL75QVDEMSEH52TYWCNVLAVCNFSM6AAAAABEB46NIWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE3DGNRVGAYDINA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Maxwells-Demons]

But I indeed specified clip_model in the command:
clip-retrieval back --port 1234 --indices-paths indices_paths.json --enable_hdf5 True --enableaiss_memory_mapping True --use_jit False --clip_model ViT-L/14 --use_arrow True
Or the clip_model should be specified in somewhere else?

[rom1504]

What commands did you use to build the embeddings and index ?

[Maxwells-Demons]

Inference:
time clip-retrieval inference --input_dataset="/public_bme/data/xiedian/laion400m/laion400m-data/{00000..41407}.tar" --output_folder="/public_bme/data/xiedian/laion400m/laion400m-embeds" --input_format "webdataset" --enable_metadata=True --write_batch_size=1000000 --batch_size=4096 --cache_path=None --num_prepro_workers=8
Indexing:
time clip-retrieval index --embeddings_folder="laion400m-embeds" --index_folder="index_folder" --max_index_memory_usage="160G" --current_memory_available="180G" --copy_metadata True --nb_cores 32

[rom1504]

You didn't specify the model during inference. The default is B/32

…

On Sat, Mar 2, 2024, 3:05 AM Maxwells_Ayakashi ***@***.***> wrote: Inference: time clip-retrieval inference --input_dataset="/public_bme/data/xiedian/laion400m/laion400m-data/{00000..41407}.tar" --output_folder="/public_bme/data/xiedian/laion400m/laion400m-embeds" --input_format "webdataset" --enable_metadata=True --write_batch_size=1000000 --batch_size=4096 --cache_path=None --num_prepro_workers=8 Indexing: time clip-retrieval index --embeddings_folder="laion400m-embeds" --index_folder="index_folder" --max_index_memory_usage="160G" --current_memory_available="180G" --copy_metadata True --nb_cores 32 — Reply to this email directly, view it on GitHub <#366 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAR437VZANJRHMESALXL3BTYWEXWHAVCNFSM6AAAAABEB46NIWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZUGE4TEOJYHE> . You are receiving this because you commented.Message ID: ***@***.***>

[Maxwells-Demons]

Problem solved, thanks for your generous help!