The transformer network is a sequence-to-sequence model that uses an attention mechanism to map the input sequence to the output sequence. It is a powerful model that can be used in many fields, such as machine translation, image captioning, and speech synthesis.
Text-to-speech (TTS) methods are widely used in many applications, such as voice assistants, voice chatbots, and voice-based user interfaces.
- understanding the transformer network thoroughly
- explore the application of the transformer network in TTS
- explore in other real physics questions [perhaps].
There are some materials to learn about the transformer network:
- TRANSFORMERS FROM SCRATCH
- A Mathematical Framework for Transformer Circuits
- Neural Speech Synthesis with Transformer Network
And the following are some ready-to-use code or tools:
- PaddleSpeech: an open-source toolkit on PaddlePaddle platform for a variety of critical tasks in speech and audio, with the state-of-art and influential models.
- Parakeet入门:使用WaveFlow和Transformer TTS训练语音合成模型
- SoftVC VITS Singing Voice Conversion, The singing voice conversion model uses SoftVC content encoder to extract source audio speech features.
- Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles.
- alanine dipeptide, Generate molecular configurations of alanine dipeptide using a transformer network.