Research map / Image generation / Autoregressive and latent generation
BNMusic: Blending Environmental Noises into Personalized Music
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BNMusic, which blends environmental noise into text-prompted music generated via two-step Riffusion outpainting/inpainting on the noise mel-spectrogram, followed by masking-threshold-based adaptive amplification, to reduce noise perceptibility.
Problem
Environmental noise annoys listeners in shared spaces; conventional acoustic masking needs excessive volume when the masker is misaligned with the noise (e.g., mismatched rhythm). The paper targets making noise less noticeable by blending it with personalized generated music.
Approach
Generate music conditioned on the noise's high-energy mel-spectrogram regions via a two-step outpainting-then-inpainting diffusion process (Riffusion), so the music is rhythmically and spectrally aligned with the noise, then apply a minimal adaptive gain computed from a psychoacoustic masking threshold (21 dB SMR) to strengthen masking.
Key results
- On EPIC-SOUNDS with direct outputs, BNMusic reaches FAD 7.98 and KL 1.67 versus 10.95/1.85 for MusicGen and 13.15/2.25 for Riffusion A2A.
- In a 50-sample subjective study, BNMusic scores OVL 3.67 and PER 3.84, the highest among compared methods (Riff A2A PER 3.24).
- Ablation shows the full pipeline (outpainting + inpainting + adaptive amplification) gives FAD 7.98 and KL 1.67, improving over outpainting alone (8.68 FAD, 1.89 KL).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BNMusic instead of MusicGen | Music blending with environmental noise · EPIC-SOUNDS, loudness normalized | FAD: 13.28 -> 12.86 | Helped |
| Use BNMusic instead of MusicGen | Music blending with environmental noise · ESC-50, loudness normalized | KL: 2.43 -> 2.38 | Helped |
| Use BNMusic instead of MusicGen | Music blending with environmental noise · EPIC-SOUNDS, direct outputs | FAD: 10.95 -> 7.98 | Helped |
| Use BNMusic instead of MusicGen | Music blending with environmental noise · ESC-50, direct outputs | KL: 2.33 -> 2.14 | Helped |
| Use BNMusic instead of MusicGen | Adaptive amplified audio · Human evaluation | OVL: 2.97 ± 0.34 -> 3.67 ± 0.55 | Helped |
| Use BNMusic instead of Riff A2A | Adaptive amplified audio · Human evaluation | PER: 3.24 ± 0.67 -> 3.84 ± 0.63 | Helped |
| Use BNMusic instead of Random Music | Adaptive amplified audio · Objective evaluation | FAD: 6.84 -> 7.98 | Hurt |
| Use BNMusic instead of MusicGen | Adaptive amplified audio · Objective evaluation | KL: 1.75 -> 1.67 | Helped |
| Add outpainting and inpainting (ablation) | BNMusic · Ablation study | FAD: 34.17 -> 8.00 | Helped |
| Add outpainting and inpainting (ablation) | BNMusic · Ablation study | KL: 1.78 vs -- | Helped |
| Add inpainting (ablation) | Outpainting-only BNMusic · Ablation study | FAD: 8.68 -> 8.00 | Helped |
| Add inpainting (ablation) | Outpainting-only BNMusic · Ablation study | KL: 1.89 -> 1.78 | Helped |
Surprising findings
- Random Music and MusicGen achieve better FAD than BNMusic in the adaptively amplified setting (6.84 and 7.08 vs 7.98), which the authors attribute to amplification making the music more prominent.
Limitations
- Poorly matched prompts to the noise weaken the blend and the prompt's influence.
- Mel-spectrogram and Griffin-Lim conversion introduces distortion that slightly degrades music quality.
- Slow generation (about 5 s per sample on a 4090) makes real-time use infeasible.
Experimental setup
Datasets: EPIC-SOUNDS ESC-50 MusicBench
Benchmarks: MusicBench EPIC-SOUNDS ESC-50
Metrics: FAD KL divergence OVL PER
Baselines: Riffusion audio-to-audio MusicGen melody-conditioned Random Music
Builds on: Riffusion auditory masking psychoacoustic model Griffin-Lim
Closest papers by meaning
- MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Models (CVPR 2024 · similarity 0.85)
- Customized Condition Controllable Generation for Video Soundtrack (CVPR 2025 · similarity 0.85)
- MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners (ICML 2025 · similarity 0.85)
- MusER: Musical Element-Based Regularization for Generating Symbolic Music with Emotion (AAAI 2024 · similarity 0.85)
- MusicRL: Aligning Music Generation to Human Preferences (ICML 2024 · similarity 0.85)
- V2Meow: Meowing to the Visual Beat via Video-to-Music Generation (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.