ISCA Archive Interspeech 2022
ISCA Archive Interspeech 2022

SpeechPainter: Text-conditioned Speech Inpainting

Zalan Borsos, Matthew Sharifi, Marco Tagliasacchi

We propose SpeechPainter, a model for filling in gaps of up to one second in speech samples by leveraging an auxiliary textual input. We demonstrate that the model performs speech inpainting with the appropriate content, while maintaining speaker identity, prosody and recording environment conditions, and generalizing to unseen speakers. Our approach significantly outperforms baselines constructed using adaptive TTS, as judged by human raters in side-by-side preference and MOS tests.