The objective of this project is to emperically determine whether or not the best configuration obtained from running hyperparameter optimization on a subset of a dataset will also be the best configuration when training on the full dataset. If this is the case, then using a subset of the full dataset can either allow for a much more rapid optimization process or allow for a much wider search space when optimizing hyperparameters.
I just finished a proof-of-concept trial to test out the training pipeline with raytune, please see the notebook '01 - analyze results from proof-of-concept runs on 20221019' for the discussion.
To set up the dependancies in a docker container, do:
build the image:
docker build --tag hparam_project .
start the container with a bind mount attached:
docker run -d \
--mount
--name hparam hparam_project tail -f /dev/null
open a bash terminal inside the container:
docker exec -it hparam bash
while inside the container, activate the conda environment
conda activate hparam_project
- build a simple feed forward network
- build a simple convolutional network
- implement callback handling
- conduct proof-of-concept experiment comparing learning rate optimization on mnist
- dockerize project for portability
- update requirements and add installation instructions
- rethink tran/val splits (use stratified sampling and sample 100%, 75%, 50%, 25%, and 10% of dataset)
- implement learning rate scheduling
- change learning rate hyperparameter, and other hparams in training
- implement data augmentation
linode -> object store: s3cmd put /hparam_results/lr_opt_20221109_mnist/* s3://hparam-project/hparam_results/lr_opt_20221109_mnist --recursive
https://www.linode.com/docs/products/storage/object-storage/guides/s3cmd
-
next steps 20221217
-
cifar10 + cnn2
-
mnist + resnet18
-
cifar10 + resnet18
Download files from linode: scp @:/path/to/linode/file /path/to/local/file
for a directory, use the -r flag