xingyuzhang2018/explore-transformer-network

Explore transformer network for TTS

Background

The transformer network is a sequence-to-sequence model that uses an attention mechanism to map the input sequence to the output sequence. It is a powerful model that can be used in many fields, such as machine translation, image captioning, and speech synthesis.

Text-to-speech (TTS) methods are widely used in many applications, such as voice assistants, voice chatbots, and voice-based user interfaces.

Aim of this project

• understanding the transformer network thoroughly
• explore the application of the transformer network in TTS
• explore in other real physics questions [perhaps].

Materials

There are some materials to learn about the transformer network:

• TRANSFORMERS FROM SCRATCH
• A Mathematical Framework for Transformer Circuits
• Neural Speech Synthesis with Transformer Network

And the following are some ready-to-use code or tools:

• PaddleSpeech: an open-source toolkit on PaddlePaddle platform for a variety of critical tasks in speech and audio, with the state-of-art and influential models.
• Parakeet入门：使用WaveFlow和Transformer TTS训练语音合成模型
• SoftVC VITS Singing Voice Conversion, The singing voice conversion model uses SoftVC content encoder to extract source audio speech features.
• Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles.
• alanine dipeptide, Generate molecular configurations of alanine dipeptide using a transformer network.