Is it possible to fine-tune it using a new voice?

[facebookresearch]

Is it possible to fine-tune it using a new voice?

[cndn]

Hey @rtruszkowski - I imagine the question is about getting a translation model that generates a specific voice? Then you only need to train your own vocoder. We trained the vocoder following the HifiGAN implementation in https://github.com/facebookresearch/speech-resynthesis - our multilingual version is slightly different in aux embedding.

Given a dataset you have with your own voice
(1) With our UPCOMING unit_extraction pipeline (XLSR + kmeans), extract discrete units (WIP #17 by @kauterry )
(2) Train vocoder using the library above
(3) At inference time, replace our multilingual vocoder with your vocoder

[kauterry]

#17 is merged, so this should be possible.

[blldd]

Hey @rtruszkowski - I imagine the question is about getting a translation model that generates a specific voice? Then you only need to train your own vocoder. We trained the vocoder following the HifiGAN implementation in https://github.com/facebookresearch/speech-resynthesis - our multilingual version is slightly different in aux embedding.

Given a dataset you have with your own voice (1) With our UPCOMING unit_extraction pipeline (XLSR + kmeans), extract discrete units (WIP #17 by @kauterry ) (2) Train vocoder using the library above (3) At inference time, replace our multilingual vocoder with your vocoder

Thanks for the suggestion, I'm curious about the amount of voice data. So, how many seconds of voice data do I need to collect to train a stable vocoder? And how long does the training process takes?
Further, is there any more convenient way?
Thank you for your reply :P