WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis [pdf]

Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, Najim Dehak, William Chan, Interspeech 2021

Abstract: This paper introduces WaveGrad 2, a non-autoregressive generative model for text-to-speech synthesis. WaveGrad 2 is trained to estimate the gradient of the log conditional density of the waveform given a phoneme sequence. The model takes an input phoneme sequence, and through an iterative refinement process, generates an audio waveform. This contrasts to the original WaveGrad vocoder which conditions on mel-spectrogram features, generated by a separate model. The iterative refinement process starts from Gaussian noise, and through a series of refinement steps (e.g., 50 steps), progressively recovers the audio sequence. WaveGrad 2 offers a natural way to trade-off between inference speed and sample quality, through adjusting the number of refinement steps. Experiments show that the model can generate high fidelity audio, approaching the performance of a state-of-the-art neural TTS system. We also report various ablation studies over different model configurations.

Samples from WaveGrad 2

Note: Only a single model is trained. Different rows correspond to different iterative refinement schedules for inference.

Text Weekends at twenty three fifty. Here are the match lineups for the
Colombia Haiti match.
On Friday night in Bridgeport expect a temperature of minus four degrees Fahrenheit.
Reference
WaveGrad (Linear 1000)
WaveGrad (Linear 50)
WaveGrad (Manual 6)
WaveGrad 2 (Linear 1000)
WaveGrad 2 (Linear 50)
WaveGrad 2 (Manual 6)