Training of Multitask-CNN teacher model

[wtomin]

Training of Multitask-CNN teacher model

[wtomin]

My learning rate was initially 0.0001, and I used a step lr scheduler to reduce the learning rate to 0.1 after every 3 epochs. The batch size was 64. The number of epochs trained for the teacher model was 8, and the number of epochs trained for the student model was 3.

[wtomin]

1. Which dataset did I use to train the teacher model
   Firstly, the multitask CNN model was trained on the mixed dataset, well balanced. After training, the weights of the resnet50 convolutional filters were imported in multitask CNN-RNN model, and fixed. Secondly, the multitask CNN-RNN model was trained on Aff-wild2 dataset only, because one of the external dataset is not a video dataset.
2. What parameter setting did I use to train the teacher model?
   About parameter settings, please refer to the default settings in /options, and python train.py --force_balance --name image_size_112_n_students_5 --image_size 112

BTW, I will recommend you to ask for the cropped and aligned faces of aff-wild2 dataset by emailing to [email protected] . Because that is the data I used.

[jacksoncsy]

Thanks a lot for the clarification! Yes, we are using the cropped aligned images provided the challenge organisers.
Regarding the initialisation of Multitask-CNN, I have found three options, i.e., 'ferplus', 'sfew' and 'imagenet'. May I know which one do you use? the default is 'ferplus'. Does the inistialisation makes huge difference in your training?
Thanks again!

[wtomin]

The 'ferplus' choice means that the backbone model (resnet50) was pretrained on ferplus dataset. If you choose other options, the weight initialization will be different. Especially, if you choose 'imagenet' dataset, which is an object recognition dataset, the pretrained task is very different from what we are trying to learn here.

But there might be other reasons. May I ask you provide 1) the options, it looks like

------------ Options -------------
C_adam_b1: 0.5
C_adam_b2: 0.999
F_adam_b1: 0.5
F_adam_b2: 0.999
batch_size: 64
checkpoints_dir: ./checkpoints
criterion: CE
dataset_mode: Mixed_EXPR
force_balance: True
gpu_ids: [0]
... ...
-------------- End ----------------

2 ) Your evaluation results on the validation set of Aff-wild2 dataset

[jacksoncsy]

I see, thanks!
I used the default options in the base_options.py, with the force_balance option enabled and image size to be 112.
May I ask what is the learning rate and batch_size you used? The default is batch_size 20, and lr 0.0001?

[jacksoncsy]

Thanks a lot, that's very useful!
We also found that you have two merging methods when reporting validation performance. May we know which ones did you use in your paper? Or you simply report whatever it is the best for each task?
Sorry for these many questions! We really appreciate your help, and we will certainly cite your work if there is any follow-up publication from our side.

[wtomin]

We just tried the two methods for ensembling student models: one is to average the raw outputs, the other one is to do major voting on the estimations. We found the first one had better performance than the second one for all the three tasks. So we reported the results for the first method in our paper.