akashe/multimodal-action-recognition

Code on selecting an action based on multimodal inputs. Here in this case inputs are voice and text.

multimodal-action-recognition's Issues

Help multi-modal speech action recognition

Hello, there is no audio data set file in your multi-modal speech action recognition code. Is it convenient to provide

Do I need position Embedding before CrossAttentionLayer?

Well done, I've also been studying multimodal tasks recently, if I have video frame level embedding (bs,len,dim) as well as text embedding that need to be done position Embedding before CrossAttentionLayer? Looking forward to your reply.

Where to download the wave files?

Hello, I have seen the Readme file that downloaded data should be put according to where the csv file illustrates, but I did not find out where to download these wav dataseds o(╥﹏╥)o

run train.py and report an error

Thank for your work .While running train.py,report an error.
OSError: [E050] Can't find model 'en_core_web_sm'.
Excuse me, what is the reason?

environment issues

hi，I‘d like to ask you about the detailed infomation about the required environment ,such as torch version.
I'm a rookie and still studying in this domain so forgive my stupied question.Thanks!

run train.py and report an error

Thank for your work .While running train.py,report an error.
OSError: [E050] Can't find model 'en_core_web_sm'.
Excuse me, what is the reason?