For Database Sharding Demonstration Purpose, this project was built. For database, I have used Postgres and to shard and create multiple instance I have used Docker. Flask is used to demonstrate the application that will be used with the database
- Flask
- Postgres
- Docker
Install Python and Docker in your local machine
git init
git clone https://github.com/khan-asfi-reza/db-sharding-with-flask.git
We will use the following image for our Database Nodes
cd postgres
docker build -t mypostgres .
Now we will create 3 Database Node / Server with 3 different ports
docker run --name node1 -p 5432:5432 -e POSTGRES_PASSWORD='postgres' -d mypostgres
docker run --name node2 -p 5433:5432 -e POSTGRES_PASSWORD='postgres' -d mypostgres
docker run --name node3 -p 5434:5432 -e POSTGRES_PASSWORD='postgres' -d mypostgres
pip install -r requirements.txt
Create a .env file
POSTGRES_HOST=YOUR_HOST_NAME
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DATABASE=postgres
YOUR_HOST_NAME = Your Local IP4 Address
run flask
What is Database Sharding?
- Database Sharding is a method for distributing a single dataset across multiple databases, which can then be stored on multiple machines. This allows for larger datasets to be split in smaller chunks and stored in multiple data nodes, increasing the total storage capacity of the system. MongoDB
Why and when Database Sharding?
- Easy to manage large dataset
- Build large and scalable system
- Distributed System