rishikksh20/avocodo-pytorch

Avocodo: Generative Adversarial Network for Artifact-free Vocoder

Avocodo: Generative Adversarial Network for Artifact-free Vocoder

Unofficial implementation of Avocodo: Generative Adversarial Network for Artifact-free Vocoder.

Disclaimer: It only works on config_v1.json for now and this repo build with experimentation purpose not for Production.

• For best quality speech synthesis please visit deepsync.co

Training:

python train.py --config config_v1.json

Notes:

• Avocodo uses same Generator as HiFi-GAN V1 and V2 but using different discriminators for modelling better lower and higher frequencies.
• PQMF is the crucial for both Discriminators.
• Losses are similar to HiFi-GAN.
• Performance and speed both are some what similar to HiFi-GAN.
• Avocodo far better than HiFi-GAN when it comes to synthesize unseen speaker.
• Avocodo training is around 20 % faster than HiFi-GAN also it took very less training to output excellent quality of audio.

Citations:

@misc{https://doi.org/10.48550/arxiv.2206.13404,
  doi = {10.48550/ARXIV.2206.13404},
  
  url = {https://arxiv.org/abs/2206.13404},
  
  author = {Bak, Taejun and Lee, Junmo and Bae, Hanbin and Yang, Jinhyeok and Bae, Jae-Sung and Joo, Young-Sun},
  
  keywords = {Audio and Speech Processing (eess.AS), Artificial Intelligence (cs.AI), Sound (cs.SD), FOS: Electrical engineering, electronic engineering, information engineering, FOS: Electrical engineering, electronic engineering, information engineering, FOS: Computer and information sciences, FOS: Computer and information sciences},
  
  title = {Avocodo: Generative Adversarial Network for Artifact-free Vocoder},
  
  publisher = {arXiv},
  
  year = {2022},
  
  copyright = {arXiv.org perpetual, non-exclusive license}
}

avocodo-pytorch's Issues

why add detach() ?

why do you detach()

Avocodo-pytorch/train.py

Line 130 in b298072

y_df_hat_r, y_df_hat_g, _, _ = mcmbd(y, y_g_hat.detach(), x2.detach(), x1.detach())

and not at

Avocodo-pytorch/train.py

Line 148 in b298072

y_df_hat_r, y_df_hat_g, fmap_f_r, fmap_f_g = mcmbd(y, y_g_hat, x2, x1)

@rishikksh20

about the CUDA and Torch version

Hello, may I ask about the cuda and torch version this GitHub run on? thanks a lot!

Quality vs iSTFTNet

Which do you feel has better quality, Avocodo or iSTFTNet? Any other thoughts you could share? Thanks!

PQMF cutoff value

In sub-band discriminator the cutoff values for PQMF are set as follows:
N=2: 0.25
N=4: 0.13
N=16: 0.03
N=64: 0.1

For good reconstruction with 64 subbands the optimal cutoff is closer to 0.01.
Is it purposely set to 0.1 or a possible typo by the authors?