Encoder of Transformer Components:
- Multi Head Self Attention Can be used as a layer in keras model with Functional API.
Advantages:
- Better parallelization as all time steps are processed simulataneously.
- Multiple heads imply multiple vector space representations.
- Better performance on long sequences.