sridharsola/ecgan

PyTorch Implementation of our paper 'Unmasking Your Expression: Expression-Conditioned GAN for Masked Face Inpainting' (CVPRW'23).

ECGAN

This is the repository for our work 'Unmaksing Your Expression: Expression-Conditioned GAN for Masked Face Inpainting' presented at the 5th Workshop on Affective Behaviour Analysis in the Wild part of CVPR2023 Workshops. The paper is available here. Alternatively, you can watch this short video presentation. To view the code in a notebook, check out this Colab file.

Abstract. As face masks continue to be a part of our daily lives, the challenge of reconstructing occluded faces remains relevant. While several approaches have been proposed for removing masks from neutral facial images, few have explored the use of facial expressions as a dominant feature for reconstruction of expressive faces. To address this gap, we propose an expression-conditioned GAN (ECGAN) for reconstructing masked faces with a specified expression. Our approach leverages both the binary segmentation map of the mask and an expression label to generate high-quality images. To train our ECGAN in a supervised manner, we synthesize masked images using the RAFDB dataset to create non-masked-masked pairs of images for training. We evaluate of our approach on the RAFDB test set, demonstrating its effectiveness in generating realistic images that convincingly belong to the given expression class. This is further highlighted by comparing it to a baseline model and a state of-the-art approach without expression-input.

The starter code for our conditioned GAN (vanilla UNet, loss, train, and test functions) are taken from this project.

TL;DR
ECGAN takes in a masked image, its mask binary segmentation, and an expression class, and returns an unmasked image with the expression as shown:

ecgan's Issues

Request for trained Model or Detailed Training Method

Hi,
I have followed the methodology described in your paper, but I am unable to achieve the same level of performance as reported.To better understand the specifics of your approach and ensure the correctness of my implementation, may I request the trained model or a detailed description of the training method? Thanks your help.

Importing the required libraries

MessageError Traceback (most recent call last)
in <cell line: 13>()
11
12 from google.colab import drive
---> 13 drive.mount('/content/drive')
14 # Specify the folder containing images
15 folder_path = '/content/drive/MyDrive'

3 frames
/usr/local/lib/python3.10/dist-packages/google/colab/_message.py in read_reply_from_input(message_id, timeout_sec)
101 ):
102 if 'error' in reply:
--> 103 raise MessageError(reply['error'])
104 return reply.get('data', None)of
105

MessageError: Error: credential propagation was unsuccessful
How to rectify this error , Sir please could you help me in completing this project as there is final rewiew of the project ion 30th of this month

”my_loader“ error

class RafDataset(data.Dataset):
def init(self, unmask = '/content/drive/MyDrive/FERDatasets/RAFDB/aligned',
mask_file = '/content/drive/MyDrive/Mask_RAFDB/aligned_mask',
partition = 'train', transform = None, num_classes = 7, loader = my_loader):

hello，can you tell what is the loader parameter ？