This is the Pytorch Implementation for the paper:
InfoUCL: Learning Informative Representations for Unsupervised Continual Learning
Liang Zhang, Jiangwei Zhao, Qingbo Wu, Lili Pan, Hongliang Li
IEEE Transactions on Multimedia (submitted)
Abstract: Recently, there has been a growing interest in unsupervised continual learning (UCL) as a new direction of continual learning (CL). However, most existing UCL methods have only paid attention to transferring CL strategies from supervised continual learning (SCL) to UCL, without considering whether the learned image feature suits UCL. This work highlights the problem of texture bias that has been previously ignored in existing UCL methods. To address this problem, we propose a new framework called InfoUCL, which integrates InfoDrop into UCL to encourage continual learners to learn informative image features and discard less informative ones. Our proposed InfoUCL framework can be easily combinable with most existing UCL strategies. Extensive experiments on various benchmarks have demonstrated that our InfoUCL framework achieves higher classification accuracy and superior robustness to catastrophic forgetting.
Dependencies
- Python >= 3.7.13 (Recommend to use Anaconda or Miniconda)
- PyTorch >= 1.9.1
- Option: Linux(Ubuntu 20.04.3)
Installation
First, you can clone this repo using the command:
git clone https://github.com/learninginvision/InfoUCL
Then, you can create a virtual environment using conda, as follows:
conda env create -f environment.yaml
We provide source about the datasets we use in our experiment as below:
Dataset | Dataset file |
---|---|
Split CIFAR-10 | CIFAR-10 |
Split CIFAR-100 | CIFAR-100 |
Split Tiny-ImageNet | Tiny-Imagenet |
iDigits | MNIST, SVHN, MNIST-M and SYN |
Split ColoredMNIST | ColoredMNIST |
In addition, if you do not want to modify the code, please store the dataset in the path: ../dataset/
and make sure that the folder structure under this path is as follows:
โโโ MNIST
โย ย โโโ raw
โโโ CIFAR10
โย ย โโโ cifar-10-batches-py
โโโ CIFAR100
โย ย โโโ cifar-100-python
โโโ IDIGITS
โย ย โโโ test
โย ย โโโ train
โโโ TINYIMG
โโโ processed
Run the following command to train and evaluate a model sequentially:
CUDA_VISIBLE_DEVICES={GPU_IDX} python train.py --log_dir {LOG_PATH} -c {CONFIG_PATH}
Command-line arguments are as follows:
{GPU_IDX}
: The index of GPU which you want to adopt. (default:0
){LOG_PATH}
: The path to save the log file and model checkpoints. (default:./logs/
){CONFIG_PATH}
: The path to the configuration file. (default:./configs/simsiam_c10_cassle.yaml
)
Additionally, you can change the hyperparameters in the configuration file ./configs/*.yaml
to train the model.
After training, you can get model checkpoints in the folder ./logs/exp_name
, where exp_name
is the name of the experiment (generated automatically), and the training log will be saved in the folder ./logs/exp_name/
.
After training, you can get model checkpoints in the folder ./logs/exp_name
, where exp_name
is the name of the experiment.
Then, you can re-evaluate the model using the command:
CUDA_VISIBLE_DEVICES={GPU_IDX} python test.py --log_dir {EXP_PATH}
{EXP_PATH}
is the path to the experiment folder. Additionally, the result will be saved in the folder {EXP_PATH}
.
This code is built upon divyam3897/UCL, and refers to the following four projects:
[1] InfoDrop
[2] mammoth
[3] cassle
[4] lightly
We thank the authors for releasing their code.