Research map / Image generation / Fast diffusion sampling
DiffSED: Sound Event Detection with Denoising Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reformulates sound event detection as conditional denoising diffusion over noisy event queries decoded by a DETR-style transformer decoder, outperforming SEDT on URBAN-SED and EPIC-Sounds.
Problem
Sound event detection requires predicting onset/offset times and class labels of events in audio; existing methods are discriminative (frame-level or event-level regression), and the boundary ambiguity and slow DETR-style convergence limit accuracy.
Approach
Treats SED as generative denoising: learnable event query embeddings are Gaussian-corrupted and a Transformer decoder, conditioned on a once-computed audio feature, denoises them into event onset/offset proposals and class labels, with DDIM-style sampling at inference.
Key results
- On URBAN-SED test, DiffSED reaches 43.89% event-based F1 vs 37.27% for SEDT and 69.24% segment-based F1 vs 65.21%.
- On EPIC-Sounds validation, DiffSED achieves 56.85% top-1 and 87.45% top-5 accuracy, improving top-1 by about 3.1 points over the best baseline.
- The paper reports 40+% faster convergence in training than SEDT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the discriminative SEDT decoder with diffusion-based denoising | SED on URBAN-SED · URBAN-SED test | Event-F1 (%): 37.27 -> 43.89 | Helped |
| use DiffSED instead of SSAST | EPIC-Sounds classification · EPIC-Sounds validation | Top-1 accuracy: 53.75 -> 56.85 | Helped |
| use DiffSED instead of ASF | EPIC-Sounds classification · EPIC-Sounds validation | mAUC: 0.879 -> 0.861 | Hurt |
| corrupt ground-truth boxes and project them as event queries instead of corrupting event latents (ablation) | DiffSED on URBAN-SED · URBAN-SED test | Event-F1 (%): 41.3 -> 37.29 | Hurt |
| increase event queries from 10 to 20 (ablation) | DiffSED on URBAN-SED · URBAN-SED test | Event-F1 (%): 40.78 -> 41.42 | Helped |
| increase event queries from 20 to 30 (ablation) | DiffSED on URBAN-SED · URBAN-SED test | Audio-tagging F1 (%): 76.54 -> 77.46 | Helped |
| increase event queries from 30 to 50 (ablation) | DiffSED on URBAN-SED · URBAN-SED test | Event-F1 (%): 41.3 -> 36.28 | Hurt |
| increase denoising steps from 1 to 5 (ablation) | DiffSED on URBAN-SED · URBAN-SED test | Event-F1 (%): 43.89 -> 44.35 | Helped |
| increase denoising steps from 1 to 5 (ablation) | DiffSED on URBAN-SED · URBAN-SED test | Audio-tagging F1 (%): 77.87 -> 77.07 | Hurt |
| increase denoising steps from 5 to 10 (ablation) | DiffSED on URBAN-SED · URBAN-SED test | Event-F1 (%): 44.35 -> 43.50 | Hurt |
| set noise scaling factor to 0.4 rather than 0.3 (ablation) | DiffSED on URBAN-SED · URBAN-SED test | Event-F1 (%): 41.3 -> 43.89 | Helped |
| set noise scaling factor to 0.4 rather than 0.3 (ablation) | DiffSED on URBAN-SED · URBAN-SED test | Audio-tagging F1 (%): 77.46 -> 77.87 | Helped |
Surprising findings
- Single-step decoding gives the best audio tagging performance, while multistep denoising (5 steps) improves event and segment F1 only slightly.
Limitations
- Event-based F1 gain is reported on one dataset only with a ResNet-50 encoder; EPIC-Sounds mAUC is lower than ASF.
- Multistep denoising gives marginal gains and degrades beyond 5 steps, so the generative benefit is limited.
- The paper's diffusion formulation is loosely specified (e.g., equations and noise schedule description are inconsistent).
Experimental setup
Datasets: URBAN-SED EPIC-Sounds
Benchmarks: URBAN-SED EPIC-Sounds
Metrics: event-based F1 segment-based F1 audio tagging F1 top-1 accuracy top-5 accuracy mCA mAP mAUC
Baselines: CRNN-CWin Ctrans-CWin SEDT ASF SSAST
Builds on: DETR DDIM DiffusionDet SEDT
Closest papers by meaning
- ZeroSep: Separate Anything in Audio with Zero Training (NeurIPS 2025 · similarity 0.85)
- A Versatile Diffusion Transformer with Mixture of Noise Levels for Audiovisual Generation (NeurIPS 2024 · similarity 0.85)
- DiffAR: Denoising Diffusion Autoregressive Model for Raw Speech Waveform Generation (ICLR 2024 · similarity 0.84)
- Denoising Task Difficulty-based Curriculum for Training Diffusion Models (ICLR 2025 · similarity 0.84)
- DTF-AT: Decoupled Time-Frequency Audio Transformer for Event Classification (AAAI 2024 · similarity 0.84)
- MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generation (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.