zcemycl/aws-chromadb-terraform

Deployment of chromadb into AWS resources through terraform

aws-chromadb-terraform

The repository to deploy chromadb via terraform into aws cloud infrastructure, using API Gateway, Cloud Map, Service Discovery, NLB, EFS, ECS Fargate and VPN.

Architecture 3	Architecture 4
Architecture 5

Architectures

1. Vanilla public ec2 instance [code]
   • Translated from the cloudformation template here to terraform.
2. Public ec2 instance with API gateway [code]
   • Modified from the youtube video How to run a Chroma Vector Database locally and on AWS! | EASY MODE
   • Additional cloudwatch to view api gateway deployment.
3. Private ec2 instance with Network Load Balancer and API Gateway [code]
   • Modeified from the youtube video Deploy a PRIVATE Chroma Vector DB to AWS | Step by step tutorial | Part 2
   • Additional cloudwatch to view api gateway deployment.
   • Additional public ec2 to view docker logs within private ec2.
   • WARNING: Can take 10 mins to deploy due to VPC Link !!!
4. (RECOMMENDED) Private ecs fargate with Network Load Balancer, EFS and API Gateway [code]
   • Fargate to manage docker containers.
   • Elastic File System for persistent volume of docker.
   • Cloudwatch Logs to store api gateway deployment messages and docker logs.
   • WARNING: Can take 10 mins to deploy due to VPC Link !!!
5. (RECOMMENDED) Private ecs fargate with EFS, Cloud Map and VPN Endpoint. [code]
   • Private microservices discoverable by Cloud Map.
   • Public access only with VPN client.
   • [IMPORTANT] Read [this] to set up your vpn properly.
   • WARNING: Can take 10 mins to deploy due to VPN Networking !!!

How to deploy?

cd architectures/{architecture-directory}
terraform init
terraform plan
terraform apply -auto-approve

How to debug?

1. Remote access to ec2 instance.

   # architecture 1,2
   ssh -i ssh-chroma.pem ec2-user@{public-chroma-ip}
   sudo docker logs `sudo docker ps | grep chroma | awk '{ print $1 }'`
   # architecture 3
   ssh -i ssh-chroma.pem ec2-user@{public-backdoor-ip}
   ssh -i ssh-chroma.pem ec2-user@{private-chroma-ip}
   sudo docker logs `sudo docker ps | grep chroma | awk '{ print $1 }'`
   # architecture 4,5 (not applicable)

2. Curl

   # architecture 1.2
   curl http://{public-chroma-ip}:8000/api/v1/heartbeat
   # architecture 2,3,4
   curl --location --request GET \
       'https://*******.execute-api.eu-west-2.amazonaws.com/v1/api/v1/heartbeat' \
       --header 'x-api-key: ****'
   # architecture 5 (vpn on)
   curl http://test.service.internal:8000/api/v1/heartbeat

3. Postman (architecture 2,3,4)
   • GET
     • (architecture 1,2) http://{public_chroma_ip}:8000/api/v1/heartbeat
     • (architecture 2,3,4) https://******.execute-api.eu-west-2.amazonaws.com/v1/api/v1/heartbeat
     • (architecture 5 (vpn on)) http://test.service.internal:8000/api/v1/heartbeat
   • Authorization Type: API Key (architecture 2,3,4)
     • Key: x-api-key
     • Value: {api key}
4. Python requests

   import requests
   # architecture 1,2
   uri = 'http://{public_chroma_ip}:8000/api/v1/heartbeat'
   headers = {}
   # architecture 2,3,4
   uri = 'https://*****.execute-api.eu-west-2.amazonaws.com/v1/api/v1/heartbeat'
   headers = {"x-api-key": "****"}
   # architecture 5 (vpn on)
   uri = 'http://test.service.internal:8000/api/v1/heartbeat'
   headers = {}
   # ------
   response = requests.get(uri, headers=headers)
   print(response.text)

5. Read Cloudwatch logs
   • from API-Gateway-Execution-Logs_xxxxxx. (architecture 2,3,4)
   • from chroma-container-logs. (architecture 4, 5) -> replace step 1
6. Python sdk from chroma-core

   import chromadb
   
   # architecture 1,2
   hostname = '{public_chroma_ip}'
   ssl, port, headers = False, 8000, {}
   # architecture 2,3,4
   hostname = '*****.execute-api.eu-west-2.amazonaws.com/dev'
   ssl, port = True, ""
   headers = {"x-api-key": "****"}
   # architecture 5 (vpn on)
   hostname = "test.service.internal"
   ssl, port, headers = False, 8000, {}
   # ------
   
   client = chromadb.HttpClient(
       host=hostname, # don't include http or https
       ssl=ssl, port=port,
       headers=headers
   )
   print("Heartbeat: ", client.heartbeat())
   print("List collections: ", client.list_collections())

aws-chromadb-terraform's Issues

Volume EFS

I am trying to implement architecture 5

I have a problem with the ECS and EFS. The data is not saved in index_data. I don't know where it is. can you help me, please

why not deploy on Fargate instead of ec2?

Ssh is depreciated and goes against devops and gitops principles. Please make a version that builds the apps with Docker and deploys on Fargate.

Error: Warning: Argument is depreciated

Whenever I type terraform plan or terraform apply -auto-approve I get this error message:

❯ terraform apply -auto-approve
╷
│ Warning: Argument is deprecated
│
│ with module.public_subnet_network.aws_eip.nat_gateway,
│ on ../../modules/subnets/main.tf line 36, in resource "aws_eip" "nat_gateway":
│ 36: vpc = true
│
│ use domain attribute instead
│
│ (and 2 more similar warnings elsewhere)
╵
╷
│ Error: Unsupported attribute
│
│ on instances.tf line 60, in resource "aws_instance" "chroma_instance":
│ 60: module.security_groups.chroma_instance_sg_id
│ ├────────────────
│ │ module.security_groups is a object
│
│ This object does not have an attribute named "chroma_instance_sg_id".
╵
╷
│ Error: Unsupported attribute
│
│ on instances.tf line 89, in resource "aws_instance" "backdoor_instance":
│ 89: module.security_groups.backdoor_instance_sg_id
│ ├────────────────
│ │ module.security_groups is a object
│
│ This object does not have an attribute named "backdoor_instance_sg_id".