Comments (5)
I fixed this by changing the id to uuid so it doesnt overwrite the previously stored files when uploading new stuff.
Will add a pr later today.
from vault-ai.
Any updates on this issue? @AitoD Can you please elaborate on what you changed to use uuids rather than ids? I am running into the overwriting issue when trying to compile a knowledge-base.
from vault-ai.
I tried changing a few things in the pinecone.go and postapi files but kept experiencing issues with npm start giving me an error of.
postapi\pinecone.go:29:14: uuid.New undefined (type string has no field or method New) ... error
and
postapi\pinecone.go:97:2: syntax error: non-declaration statement outside function body ... error
from vault-ai.
Got it. I think this works:
diff --git a/vault-web-server/postapi/pinecone.go b/vault-web-server/postapi/pinecone.go
index 2d8f1bd..0f9fae3 100644
--- a/vault-web-server/postapi/pinecone.go
+++ b/vault-web-server/postapi/pinecone.go
@@ -11,6 +11,7 @@ import (
"math"
"net/http"
"strconv"
+ googleid "github.com/google/uuid"
)
type PineconeVector struct {
@@ -27,8 +28,9 @@ func upsertEmbeddingsToPinecone(embeddings [][]float32, chunks []Chunk, uuid str
vectors := make([]PineconeVector, len(embeddings))
for i, embedding := range embeddings {
chunk := chunks[i]
+ myuuid := googleid.NewString()
vectors[i] = PineconeVector{
- ID: fmt.Sprintf("id-%d", i),
+ ID: fmt.Sprintf("id-%s", myuuid),
Values: embedding,
Metadata: map[string]string{
"file_name": chunk.Title,
Basically import google uuid library with a different name so as not to be overwritten by func param, and then use it to generate a uuid.
I don't have the code handy but it seems to me that the code that needs to be changed is:
ID: fmt.Sprintf("id-%d", i)
According to ChatGPT one way to generate a UUID in go is :
package main
import (
"github.com/satori/go.uuid"
)
func main() {
myuuid, err := uuid.NewV4()
}
So that may do the trick. I'll try it later when I have access to the code.
from vault-ai.
Got it. I think this works:
diff --git a/vault-web-server/postapi/pinecone.go b/vault-web-server/postapi/pinecone.go index 2d8f1bd..0f9fae3 100644 --- a/vault-web-server/postapi/pinecone.go +++ b/vault-web-server/postapi/pinecone.go @@ -11,6 +11,7 @@ import ( "math" "net/http" "strconv" + googleid "github.com/google/uuid" ) type PineconeVector struct { @@ -27,8 +28,9 @@ func upsertEmbeddingsToPinecone(embeddings [][]float32, chunks []Chunk, uuid str vectors := make([]PineconeVector, len(embeddings)) for i, embedding := range embeddings { chunk := chunks[i] + myuuid := googleid.NewString() vectors[i] = PineconeVector{ - ID: fmt.Sprintf("id-%d", i), + ID: fmt.Sprintf("id-%s", myuuid), Values: embedding, Metadata: map[string]string{ "file_name": chunk.Title,
Basically import google uuid library with a different name so as not to be overwritten by func param, and then use it to generate a uuid.
I don't have the code handy but it seems to me that the code that needs to be changed is:
ID: fmt.Sprintf("id-%d", i)
According to ChatGPT one way to generate a UUID in go is :
package main import ( "github.com/satori/go.uuid" ) func main() { myuuid, err := uuid.NewV4() }
So that may do the trick. I'll try it later when I have access to the code.
Would I be putting this in the pinecone.go file? I received a syntax error so I imagine it's because I'm doing something wrong haha
from vault-ai.
Related Issues (20)
- How to update OPEN AI API KEY? HOT 4
- pdfinfo not found
- PDF not analyzed, always answers with "Great, how may I assist you today?"
- Error: 413 | Request body size exceeds the limit HOT 1
- Error extracting text from PDF exec: "pdftotext": executable file not found in %PATH% HOT 2
- Error: NetworkError when attempting to fetch resource. HOT 1
- Error 401 HOT 2
- Documents not saving to knowledge base? HOT 2
- Do you support uploading code libraries (such as GitHub links) so that they can read and understand the source code?
- Am I supposed to see "Enter your OpenAI API key here.." field on http://localhost:8100/ ? HOT 1
- Stuck in the step Installing './vault-web-server' dependencies
- openAI embedding error timeout HOT 1
- Error getting embeddings: error, status code: 429, HOT 3
- Add documentation that explains how to hook up Qdrant vector db
- @babel/plugin-proposal-class-properties is no longer maintained
- Feature Request: Use Mistral or Llama instead of OpenAI HOT 1
- Can't get past injestion
- No Summary of content, just of style HOT 1
- I want to use the article image as the cover image Sngine blog
- How do I require uploading images when posting to a blog?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vault-ai.