visipedia/ssw60

Sapsucker Woods 60 Audiovisual Dataset

Visipedia

Visipedia Webpage

ssw60's Issues

Inquiry about the audio augmentation and evaluation setting for audio-vidual mid-fusion

Hi, thanks for your work on AV FGC task. I'd like to inquire about some experiment details in your paper:

1. In 4.1-Audio-Modality in your paper, you use logit average as evaluation strategy, but in 4.1-Audio-Visual-fusion part, you might not use the same averaging strategy for mid-fusion because they conflict, and I'd like to know how you process the audio in mid-fusion part. Do you use only one spectrogram in the mid-fusion part? If so, is the uni-modal and multi-modal result comparable?
2. In mid-fusion part you use MBT as a SOTA fusion method, I'd like to know do you ever try on other simple mid-fusion method like concatenation、summation or Gated.
3. For audio agmentation method, is it possible that you could include relevant code when you release your experiment pipeline. I would appreciate that.