Comparing python and go gives different embeddings

[ThomasAlxDmy]

Comparing python and go gives different embeddings

[matteo-grella]

Sorry for my late reply, I’ll back to you shortly.

[fkrasnov]

Hey, Thomas.

just a thought. May be it helps.
Try L2 norm for the Go-embeddings.

I have done:

a = np.array([0.073092446, 0.15883854, -0.23150429, ..])
a / np.sqrt(np.sum(np.square(a)))

Output:

array([ 1.15892480e-02, 2.51848081e-02, -3.67064008e-02, ..])

It is np.allclose() to Python embeddings

Best
Fedor

[acheong08]

What is the equivalent for Go?

I try

func l2norm(a []float64) []float64 {
	result := make([]float64, len(a))
	sum := 0.0
	for _, v := range a {
		sum += v * v
	}
	for i, v := range a {
		result[i] = v / math.Sqrt(sum)
	}
	return result
}

but if gives different results.

Is it important?

[corani]

This gives the same result as in Python:

fmt.Printf("%#v - %d\n\n, result.Vector.Normalize2(), result.Vector.Size())