Cosine similarity between embeddings

[abelriboulot]

Cosine similarity between embeddings

[abelriboulot]

Hey @ankitkr3 !
So one easy way to calculate cosine similarity between two sentences which I have used in the past it to simply compute the encoder embeddings for each of them, average it across tokens, and then compute the cosine sim. I've added an example in examples/compute_cosine_similarity.py !

[ankitkr3]

@abelriboulot Hey!
Thanks, i will check it out.

[ankitkr3]

Hi @abelriboulot
The accuracy is not as good as bert models, how can we increase the accuracy for better contextualized embeddings?

[abelriboulot]

Hey @ankitkr3 ,
Could you explain what you mean when you talk about accuracy? For embeddings often times adding the prefix of a task ("summarize: ") will help (and I'd be curious whether that improves things for you). Otherwise there are other models made to produce nicely structured embedding spaces like that one: https://tfhub.dev/google/universal-sentence-encoder-multilingual-qa/3

[ankitkr3]

@abelriboulot i am comparing the similarity between two paragraphs here, and i want to achieve high semantic calculations.

[abelriboulot]

So when you do the cosine similarity you basically measure how far away two embeddings are. So it's not a very relevant measure in the abstract. The way that you can go about figuring out whether things are well distanced are not is by evaluating how far two similar embeddings are compared to one of those embeddings vs. something that is dissimilar. So say for instance: "The sky is blue" and "The weather is clear" should be closer to each other than "The sky is blue" and "I ate pudding today".

[ankitkr3]

@abelriboulot yes, but how can we achieve better accuracy for such semantic comparisons, can we train the model on such task and then just output the embeddings ?