AudioSR: Versatile Audio Super-resolution at Scale



Haohe Liu1, Ke Chen2, Qiao Tian3, Wenwu Wang1, Mark D. Plumbley1

1CVSSP, University of Surrey

2University of California San Diego

3Speech, Audio & Music Intelligence (SAMI), Bytedance


[Paper on ArXiv]    [Code on GitHub]    [Discord Community]

Abstract

Audio super-resolution is a fundamental task that predicts high-frequency components for low-resolution audio, enhancing audio quality in digital applications. Previous methods have limitations such as the limited scope of audio types (e.g., music, speech) and specific bandwidth settings they can handle (e.g., 4 kHz to 8 kHz). We introduce a diffusion-based generative model, AudioSR, that is capable of performing robust audio super-resolution on versatile audio types, including sound effects, music, and speech. Specifically, AudioSR can upsample any input audio signal within the bandwidth range of 2 kHz to 16 kHz to a high-resolution audio signal at 24 kHz bandwidth with a sampling rate of 48 kHz. Extensive objective evaluation on various audio super-resolution benchmarks demonstrates the strong result achieved by the proposed model. In addition, our subjective evaluation shows that AudioSR can acts as a plug-and-play module to enhance the generation quality of a wide range of audio generative models, including AudioLDM, Fastspeech2, and MusicGen.



Figure 1: The AudioSR architecture. The replacement-based post-processing aims to preserve the original lower-frequency information in the model output.

Table 1: Subjective evaluation shows that applyin AudioSR for audio super-resolution on the output of audio generation models can significantly enhance the perceptual quality.



Audio Super-Resolution across Different Models

Speech:

8 kHz Sample NVSR-DNN NVSR-ResUNet AudioSR Ground-Truth

Sound Effect:

8 kHz Sample NVSR-DNN NVSR-ResUNet AudioSR Ground-Truth

Music:

8 kHz Sample AudioSR Ground-Truth

AudioSR on Improving Generative Models

FastSpeech 2 (Text-to-Speech, 22.05 kHz to 48 kHz):

The "Generation" column is the model output (e.g., from MusicGen) after the pre-processing step introduced in the paper.

Generation AudioSR Improvement

AudioLDM (Text-to-Audio, 16 kHz to 48 kHz):

Generation AudioSR Improvement

MusicGen (Text-to-Music, 32 kHz to 48 kHz):

Generation AudioSR Improvement

More demos are on their way. Stay tuned for more updates.