Gibberish audio on preliminary training

[sh-lee-prml]

Gibberish audio on preliminary training

[hayeong0]

What does 'unintelligible speech' mean? Can I see your training logs or Tensorboard?

We have used about 60 hours of Hindi data from LIMMITS and have experience using phonemizer. Have you checked if tokens were properly extracted during the training and inference stages?

[sh-lee-prml]

Hi

I have attached tensorboard loss curves for our TTV v1 model which was trained with LibriTTS-960 dataset.
we used 4x GPUs with 128 batch size (32 per GPU).

.

How about the ctc loss curve you trained? Our checkpoint is from 930k steps.

and

I actually do not know Hindi language well... but I think Phonemizer may not be good for Hindi Language. In this case, how about using other tokenizer?

[Pranjalya]

We have used phonemizer as well, and from past experience, it works decently for Hindi as well.
Here are my logs:

"unintelligible" means like it sounded like it was speaking clearly but nothing related to the text and not in the language. But again, it was just with 20k steps checkpoint.

[rishikksh20]

@hayeong0 from how many steps onward we start getting some audible voice when train TTV from scratch ?

[Pranjalya]

Just for reference, the audio from 20k steps.

hierspeech_mms.mp4

[sh-lee-prml]

Here is our results from 10k, 20k, 50k, 100k, 200k, 950k. (with hierspeech synthesizer v1)

I have attached audio for some text and speaker of libritts-test-clean.

Link

When using LibriTTS dataset, the 10k steps model can synthesize an audible speech.

Thanks!

[Pranjalya]

Thank you very much, it helped.