kirill-markin/split-documents-upload-to-pinecone

Divide documents and upload text segments to Pinecone with the script.

Kirill Markin — Founder of ozma.io, Head of AI Lab

Contacts	Social	Roles
Email: [email protected]	Linkedin @kirill-markin	Founder of ozma.io
Telegram: @kirmark	Twitter @kirill_markin_	Head of AI Lab at SOAX
WhatsApp: [phone]	Youtube @kirill-markin	Judge at MassChallenge
	Instagram @kirill.markin.kira	Mentor at Founder Institute
	Facebook @kirill.markin.kira	Mentor at MentorCruise

📆 Book a 15-minute meeting
👉 Check media mentions, conferences and articles

split-documents-upload-to-pinecone's Issues

pinecone.core.client.exceptions.ServiceException: (503)

Number of documents: 1
Number of splited documents: 337
Dimension in embeddings: 1536
Index 'infostart-faq' deleted.
pinecone.describe_index(index_name):, IndexDescription(name='infostart-faq', metric='cosine', replicas=1, dimension=1536.0, shards=1, pods=1, pod_type='p1', status={'ready': True, 'state': 'Ready'}, metadata_config=None, source_collection='')
Upload failed. Retrying in 5 seconds.
Upload failed. Retrying in 10 seconds.
Traceback (most recent call last):
  File "/Users/elenakapatsa/Repositories/split-documents-upload-to-pinecone/main.py", line 113, in <module>
    index = upload_documents_with_retry()
  File "/Users/elenakapatsa/Repositories/split-documents-upload-to-pinecone/main.py", line 111, in upload_documents_with_retry
    raise e
  File "/Users/elenakapatsa/Repositories/split-documents-upload-to-pinecone/main.py", line 103, in upload_documents_with_retry
    index = Pinecone.from_documents(docs, embeddings, index_name=index_name)
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/langchain/vectorstores/base.py", line 317, in from_documents
    return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/langchain/vectorstores/pinecone.py", line 243, in from_texts
    index.upsert(vectors=list(to_upsert), namespace=namespace)
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/core/utils/error_handling.py", line 17, in inner_func
    return func(*args, **kwargs)
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/index.py", line 150, in upsert
    return self._upsert_batch(vectors, namespace, _check_type, **kwargs)
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/index.py", line 237, in _upsert_batch
    return self._vector_api.upsert(
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/core/client/api_client.py", line 776, in __call__
    return self.callable(self, *args, **kwargs)
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/core/client/api/vector_operations_api.py", line 956, in __upsert
    return self.call_with_http_info(**kwargs)
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/core/client/api_client.py", line 838, in call_with_http_info
    return self.api_client.call_api(
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/core/client/api_client.py", line 413, in call_api
    return self.__call_api(resource_path, method,
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/core/client/api_client.py", line 207, in __call_api
    raise e
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/core/client/api_client.py", line 200, in __call_api
    response_data = self.request(
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/core/client/api_client.py", line 459, in request
    return self.rest_client.POST(url,
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/core/client/rest.py", line 271, in POST
    return self.request("POST", url,
  File "/Users/elenakapatsa/Repositories/infostart-dev/lib/python3.9/site-packages/pinecone/core/client/rest.py", line 228, in request
    raise ServiceException(http_resp=r)
pinecone.core.client.exceptions.ServiceException: (503)
Reason: Service Unavailable
HTTP response headers: HTTPHeaderDict({'content-type': 'text/plain', 'content-length': '19', 'date': 'Wed, 14 Jun 2023 11:23:48 GMT', 'server': 'envoy', 'connection': 'close'})
HTTP response body: no healthy upstream

Does anybody know how to fix this? This issue is certainly related to this, but there is no solution inside.

Pipedream: read ECONNRESET

The Pipedream automation is configured and deployed (Slack, OpenAI, Pinecone API keys added), but each message in a selected Slack chat gives me an error:

read ECONNRESET

DETAILS
    at null.__node_internal_captureLargerStackTrace (internal/errors.js:412:5)
    at null.__node_internal_errnoException (internal/errors.js:542:12)
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:209:20)

Previous steps are being completed in every attempt.