kdhht2334/elim_fer

[NeurIPS 2022] The official repository of Expression Learning with Identity Matching for Facial Expression Recognition

ELIM_FER

Optimal Transport-based Identity Matching for Identity-invariant Facial Expression Recognition (NeurIPS 2022)

PAPER | DEMO

Daeha Kim, Byung Cheol Song

CVIP Lab, Inha University

Update

• 2022.09.20: Initialize this repository.

Requirements

• Python (>=3.8)
• PyTorch (>=1.7.1)
• pretrainedmodels (>=0.7.4)
• Wandb
• Fabulous (terminal color toolkit)

To install all dependencies, do this.

pip install -r requirements.txt

Datasets

1. Download four public benchmarks for training and evaluation (please download after agreement accepted).

• AffectNet
• Aff-wild
• Aff-wild2
• AFEW-VA

(For more details visit website)

2. Follow preprocessing rules for each dataset by referring pytorch official custom dataset tutorial.

Training

Just run the below script!

chmod 755 run.sh
./run.sh <method> <gpu_no> <port_no> 

• <method>: elim or elim_category
• <gpu_no>: GPU number such as 0 (or 0, 1 etc.)
• <port_no>: port number to clarify workers (e.g., 12345)

• Note: If you want to try 7-class task (e.g., AffectNet), add age_script folder to your train or val. script and turn on elim_category option.

Evaluation

• Evaluation is performed automatically at each print_check point in training phase.

Demo

• Do to demo folder, and then feel free to use.
• Real-time demo with pre-trained weights

TODO

• Refactoring
• Upload pre-trained model weights
• Upload demo files
• Upload train/eval files

Note

• In case of Mlp-Mixer, please refer official repository (link)

Citation

If our work is useful for your work, then please consider citing below bibtex:

@misc{kim2022elim,
    author = {Kim, Daeha and Song, Byung Cheol},
    title = {Optimal Transport-based Identity Matching for Identity-invariant Facial Expression Recognition},
    Year = {2022},
    Eprint = {arXiv:2209.12172}
}

Contact

If you have any questions, feel free to contact me at [email protected].

elim_fer's Issues

offset of valence and arousal

Hi.

Thank you for the amazing work.

I have a question about offsets in scripts. In your code, you set offset values for valence and arousal. Can you tell me why this is necessary and how you determined these values?

Reproducing AffectNet results.

Hello!

Thanks for sharing your works.

• How can I reproduce the affectnet results?
  The script seems to require some preprocess.

• Can you share affectnet pretrained model?

Regards,
Sungguk Cha

How to create custom data for training ?

First thanks for your great work. I have a question about the dataset annotation and I need your help. Now that I have 10000 images that don't have labels. How I could label my dataset to apply to your model ? Hope that you can explain it to me.

License

Hi.
Thank you for the amazing work!
Could you clarify the license for this code? I would like to utilize this valence-arousal estimation model in my research.

Pre trained model

Hi,

I was wondering if you plan to upload the weights for a pre-trained model.
Currently, I couldn't find any.

Thanks

Estimation using my own facial image data.

Thanks for the amazing work!

For my own research, I would like to estimate valence-arousal using my own facial image data.
Which script should I use for this?
I thought this part was estimating valence and arousal, but I am not sure what "def hand_detection()" is doing.

Can you provide a script to estimate valence-arousal from a single image using pre-trained weights?