[Feature request] ColBERT V2

[filippo82]

[Feature request] ColBERT V2

[raphaelsty]

Hi @filippo82, I think it could be cool to add distillation loss of course. I plan to improve the loss function of the train module in the following weeks, there won't be any breaking change.

Before releasing the centroids algorithm of ColBERTV2 I plan to release another method to accelerate ColBERT retriever, it will be really fast and still accurate, work in progress.

[filippo82]

Hi @raphaelsty 👋🏻 thanks a lot for your reply and sorry for the slooow reply.

Let me know if there is any way I can help with testing/debugging.

[raphaelsty]

Hi @filippo82, I did release neural-cherche 1.1.0 which improve loss stability and brings better default parameters to models. Also I did release neural-tree in order to accelerate ColBERT.

Feel free to open a PR in neural-cherche if you are interested in Knowledge Distillation