aptx4869lm/forecasting-human-object-interaction-in-fpv

Get Started

• Follow INSTALL.md for all dependencies
• Download the datasets and unpack them (see ./tools/unpack_*.sh)
• Re-code the videos if necessary(see ./tools/recode_videos.sh)
• Convert the datasets into our format (see ./tools/convert_dataset.py).
• Download pre-trained models (see this link)
• Try to train on EPIC-Kitchens. You will need to set up the dataset folders and pre-trained model file in the json config

python ./train_joint.py ./exp_configs/epic-kitchens/epic-kitchens-action_trainval_i3d_csn152_vae.json --prec_bn

• [Optional]: Distributed training on multiple GPUs using synchronized batch norm. You can use the same config file. Learning rate will be automatically re-scaled.

python -m torch.distributed.launch --nproc_per_node=NUM_GPUS train.py ./exp_configs/epic-kitchens/epic-kitchens-action_trainval_i3d_csn152_vae.json -p 5 --prec_bn --sync_bn

• Evaluate a trained model. Models are saved in ./ckpt/your_exp/.

python ./eval_joint.py ./exp_configs/epic-kitchens/epic-kitchens-action_trainval_i3d_csn152_vae.json --resume your_model

• Visualize the training (requires tensorboard)

tensorboard --logdir ./ckpt/your_exp/logs