Research map / Image generation / Super-resolution
Audio Super-Resolution with Latent Bridge Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AudioLBM, a latent bridge model that performs LR-to-HR audio super-resolution in a continuous VAE waveform latent space, with frequency-aware conditioning and cascaded LBMs for upsampling beyond 48 kHz up to 192 kHz.
Problem
Audio super-resolution quality is limited because existing diffusion and bridge methods start from uninformative Gaussian priors or operate in spectrogram domains, and HR training data is scarce, especially above 48 kHz.
Approach
Compress waveforms into a continuous latent with a VAE and use a bridge process that starts from the LR-waveform latent (Dirac prior) and generates the HR latent, with frequency-aware conditioning on prior and target sampling rates and cascaded stages with prior augmentation (waveform low-pass filtering and latent blurring) for 96/192 kHz.
Key results
- On VCTK 8→48 kHz, Ours reaches LSD 0.753 and SSIM 0.893 versus AudioSR's 0.940 LSD and 0.809 SSIM, with SigMOS 3.023 vs. 2.846.
- On 48Audio 8→48 kHz, LSD falls to 1.066 from AudioSR's 1.468 and ViSQOL rises to 3.281 from 3.156.
- For 16→192 kHz, the cascaded pipeline lowers LSD to 1.365 from 1.913 for a directly trained any-to-192 model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| frequency-aware training (input- and target-frequency awareness) plus dataset filtering versus no filtering (ablation) | AudioLBM any-to-48 kHz · ESC-50 and SDS 16→48 kHz | LSD: ESC-50 LSD 1.366 to 0.994; SDS 1.461 to 1.124 | Helped |
| training only on native 48 kHz data instead of the any-to-any scheme (ablation) | AudioLBM · ESC-50 and SDS 16→48 kHz | LSD: ESC-50 1.127 vs 0.994; SDS 1.198 vs 1.124 | Hurt |
| removing target-frequency awareness (keeping input awareness) (ablation) | AudioLBM · ESC-50 16→48 kHz | LSD: 1.022 vs 0.994 | Hurt |
| adding cascaded LBMs (48 then 96 then 192 kHz) versus direct any-to-192 kHz training | AudioLBM cascade beyond 48 kHz · 16→192 kHz on 192Audio and 192Music | LSD: 1.913 direct vs 1.365 cascaded | Helped |
| cascaded 96 kHz LBM versus direct any-to-96 kHz training | AudioLBM beyond 48 kHz · 16→96 kHz on 96Audio and 96Music | ViSQOL: 3.010 direct to 3.330 cascaded | Helped |
| latent bridge model (noise-predicting, latent-to-latent) instead of AudioSR diffusion baseline | any-to-48 kHz SR · VCTK 8→48 kHz speech | LSD: 0.753 vs 0.940 (AudioSR) | Helped |
| latent bridge model instead of AudioSR baseline | any-to-48 kHz SR · 48Audio 8→48 kHz | ViSQOL: 3.281 vs 3.156 | Helped |
| latent bridge model instead of AudioSR baseline without low-frequency replacement | any-to-44.1 kHz SR · ESC-50 and SDS 16→44.1 kHz | LSD: ESC-50 0.999 vs 1.292; SDS 1.160 vs 1.352 | Helped |
| cascading 96 kHz intermediate stage with prior augmentation (filtering and latent blur) versus no cascade augmentation on 96 kHz data | AudioLBM cascade · 16→96 kHz on 96Audio and 96Music | LSD: direct 1.406 vs cascaded 1.216 | Helped |
| KL weight reduced to 0 with latent scaling (compression rate 512) versus standard KL in VAE (ablation) | VAE for AudioLBM · VCTK reconstruction and generation | LSD: KL 0 LSD 0.651 vs KL 1e-3 LSD 0.664 (reconstruction) | Helped |
| lower compression rate (512 vs 2048) in latent space (ablation) | VAE for AudioLBM · VCTK generation | LSD: 2048: 0.730; 512: 0.710; 256: 0.687 (generation) | Mixed |
Surprising findings
- Lower VAE compression rates and near-zero KL weight improve SR reconstruction, contrary to typical diffusion-based latent design.
- Pretraining the VAE on 48 kHz audio transfers well to 96 and 192 kHz, outperforming direct high-rate pretraining in some settings.
Limitations
- Low-frequency speech noise can be amplified and mistaken for texture in sound effects when trained on zero-shot diverse data, slightly degrading perceptual quality.
- Lack of error bars or statistical significance reporting.
- Code is not available at submission time.
Experimental setup
Datasets: VCTK ESC-50 Song-Describer-Dataset
Benchmarks: VCTK ESC-50 Song-Describer-Dataset 48Audio (internal)
Metrics: LSD SSIM ViSQOL SigMOS
Baselines: AudioSR A2SB Bridge-SR NU-Wave 2 FlowHigh Frepainter AP-BWE NVSR
Builds on: Schrödinger bridge DiT A2SB Bridge-SR Stable-Audio-Open
Closest papers by meaning
- Inference-time Scaling for Diffusion-based Audio Super-resolution (AAAI 2026 · similarity 0.89)
- SFSRNet: Super-resolution for Single-Channel Audio Source Separation (AAAI 2022 · similarity 0.87)
- Latent Space Super-Resolution for Higher-Resolution Image Generation with Diffusion Models (CVPR 2025 · similarity 0.85)
- Fast Timing-Conditioned Latent Audio Diffusion (ICML 2024 · similarity 0.83)
- RFWave: Multi-band Rectified Flow for Audio Waveform Reconstruction (ICLR 2025 · similarity 0.83)
- Multi-band Frequency Reconstruction for Neural Psychoacoustic Coding (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.