shinhyeokoh/rwen

RWEN-TTS: Relation-aware Word Encoding Network for Natural Text-to-Speech Synthesis

• Implements the model described in the following paper "RWEN-TTS: Relation-aware Word Encoding Network for Natural Text-to-Speech Synthesis" (AAAI 2023).

An example of SRE (gif)

An example of AWRE (gif)

Our Codes

Due to the policy of our previous company, our codes are partially released.

We have only released core codes of RWEN except the TTS Model (e.g. fastpitch).

However, released codes related to our contributions are included in the src folder and includes the dataset function, collate function, and model class for easy integration into the TTS Model.

Codes are based on the fastpitch 1.1 repository and could be reproducible with proper integration of that repo and our source.

Since RWEN-TTS(Fastpitch w/ AL) needs text processing by phonemes, we need to take this into account. For example, for the fastpitch 1.1 repository, use english_cleaners_v3 function from src/common/cleaners.py and english_phonemized from src/common/symbols.py.

rwen's Issues

Model scheduled for release march 2023 but still no models?

Can you kindly explain how to adapt this to any ither (for example on top of a coqui-tts which uses vits) or just release the models as they were supposedly Scheduled to be released so we can start using this industrially. Thanks a lot <3

Language Model

Hey!

In the paper you mention that you had a relative CMOS increase of 0.22 when using ELECTRA vs BERT as the pretrained language model.

Do you have any audio samples demonstrating text inputs that have noticeable improvement?

Code Release

Hey!

Is there a timeline on code release for your work?