Do I need position Embedding before CrossAttentionLayer?

[akashe]

Do I need position Embedding before CrossAttentionLayer?

[akashe]

I didn't understand the question completely. If you meant should you add positional embeddings to your video embeddings then yes, you should. It should work better if you add a positional embedding to your video embeddings because there is no notion of sequential information in video embeddings by default. With positional info, the attention mechanisms can extract features that are changing over time. Hope, that helped.

[Breeze-Zero]

I didn't understand the question completely. If you meant should you add positional embeddings to your video embeddings then yes, you should. It should work better if you add a positional embedding to your video embeddings because there is no notion of sequential information in video embeddings by default. With positional info, the attention mechanisms can extract features that are changing over time. Hope, that helped.

Thanks, I use efficientNet to get the video frame feature and Bert to get the text feature. Then, I want to refer to your CrossAttentionLayer for modal fusion, so I am confused about whether position Embedding is required