numediart/emotion-eeg

Public Models considered for emotion estimation from EEG

emotion-eeg's Issues

about the code

I used your code to run, but it reported an error, it shows such an error in the classifier section：
“”RuntimeError: Given groups=1, weight of size [32, 5, 3, 3], expected input[144, 3, 32, 32] to have 5 channels, but got 3 channels instead“”
please reply me if you see it, thank you

Runtime error on DEAP Dataset

Hi,

I've tried to run the Sal_model.py on DEAP, but apart from fixing the number of output units of the classifier (using 4 for the valence/arousal quadrants) I can't get it to work because of an error:

RuntimeError: shape '[168, 5, 1, -1]' is invalid for input of size 43352064
The size is the product of 8064x32x168 (at least that's what it seems), of time samples x channels x (batch_size=128 + 40?)

The line is the following:
https://github.com/numediart/Emotion-EEG/blob/master/Models_DEAP.py#L530

Also I implemented the feature generation part myself, as it was incomplete (only the sensor locations code conversion is in the repo), by using "mne.time_frequency.psd_array_welch" and then averaging the power values of 5 frequency bins that are mentioned in the DEAP paper (for a total of 5 image channels which should be correct).

Best Regards.

Trouble Reproducing Results

Hello,

I was trying to reproduce the results you reported in your paper regarding the DEAP dataset. While doing so, I encountered several issues and it would be great if you could briefly comment on them.

1. Input to the Model

In your paper, you describe that your framework consists of two models: a CNN designed to process EEG images and a H-RNN that accounts for information contained within different spatial levels.
However, it is not totally clear how the frequential array and the respective EEG images are obtained. Following the approach of @ale-pavel, one way would be to compute the PSD for each channel and each frequency band. For the DEAP dataset and the 5 common frequency bands, this would result in a frequential array $x$ of shape $x \in \mathbb{R}^{32 \times 5}$.
Using your image generation function, and the location mapping file, respective EEG images of size $\hat{x} \in \mathbb{R}^{5 \times 32 \times 32}$ could be generated.

Is this approach in line with your work or did you obtain your inputs differently?

2. Training and Evaluation

Following the above mentioned approach to generate inputs for your framework, I tried to train your model using the Sal_Model.py script.
It appears that the loss becomes nan after a few epochs. This might be related to the following line of code (168):

saliency = (saliency/saliency.max()+0.05)*2

as the maximum of the saliency matrix becomes 0 which results in a division by zero. Fixing this line using:

saliency = (saliency/(saliency.max()+0.05))*2

allows me to train your model. However, I fail in reproducing the results you report in your paper, i.e. $78.47$% $\pm 4.9$% accuracy using LOSO validation. Instead, I obtain accuracies that respond to a model predicting always the majority class (considering the label imbalance for each subject). Could you maybe outline how you trained your model?

The tutorial

can you share the address of the research report or paper? Thanks a lot

the size of array

Why is the size of the array (500,31,3) instead of (500,32,3)?