This is a template set up to test the performance of hashing and change push processes for a million records using SQLAlchemy ORM and postgres.
To begin with ensure that all python libraries contained within requirements.txt
are installed in the environment you will be running this or within a venv.
- Source data to test this process is created by running fake2db with the following command:
fake2db --rows 1000000 --host localhost --port 5432 --db postgresql --user <postgres_username> --name perf_test
-
Then use sqlaodegen to generate the declarative classes for the tables created by fake2db and deposit in
/models
folder -
Run the following command to create the output tables for this process within the postgres db:
python3 perf_test.py CreateDatabase dest