Publications

GR0: Self-Supervised Global Representation Learning for Zero-Shot Voice Conversion

ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Publication date: April 14, 2024

Yunyun Wang, Jiaqi Su, Adam Finkelstein, Zeyu Jin

Research in generative self-supervised learning (SSL) has largely focused on local embeddings for tokenized sequences. We introduce a generative SSL framework that learns a global representation that is disentangled from local embeddings. We apply this technique to jointly learn a global speaker embedding and a zero-shot voice converter. The converter modifies recorded speech to sound as if it were spoken by a different person while preserving the content, using only a short reference clip unavailable to the model during training. Listening experiments conducted on an unseen dataset show that our models significantly outperform SOTA baselines in both quality and speaker similarity for various datasets and unseen languages.


Research Areas:  Adobe Research iconAI & Machine Learning Adobe Research iconAudio