Research map / Video / Action recognition
Boundary Denoising for Video Activity Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DenoiseLoc, an encoder-decoder video activity localizer trained with a boundary-denoising task that jitters ground-truth spans so the decoder learns to refine boundaries, yielding faster convergence and SOTA on MAD and QVHighlights.
Problem
Video activity localization suffers from ambiguous, subjective temporal boundaries, making exact start/end prediction hard and slow to train.
Approach
Adapt denoising-based DETR training (DN-DETR/DINO/DiffusionDet style) to the temporal dimension: noisy ground-truth spans are fed to the decoder alongside learnable proposals, and the decoder learns to denoise them layer by layer; the denoiser is disabled at inference. Uses DynamicConv with RoI-aligned proposal features and only ~30 queries without NMS.
Key results
- On QVHighlights test, DenoiseLoc reaches 59.27 R@1@0.5 and 42.96 average mAP, ahead of UMT (36.12 avg mAP).
- On TACoS, DenoiseLoc achieves higher R@1 than VLG-Net at all IoU thresholds (e.g., 35.89 vs 34.19 at IoU=0.5) but lags on R@5.
- On THUMOS14, with denoising mAP@0.5 rises from 62.62 to 64.26, with only 30 queries and no NMS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add boundary-denoising training (ablation) | DenoiseLoc on QVHighlights val (with self-attention and Dynamic Conv) · QVHighlights val | mAP Avg.: 44.56 vs 42.95 without denoise | Helped |
| add self-attention in decoder (ablation) | Moment-DETR-based decoder on QVHighlights val · QVHighlights val | mAP Avg.: 41.08 vs 32.30 (no-component baseline row) | Helped |
| add Dynamic Conv instead of average fusion of feature and proposal (ablation) | Decoder without self-attention on QVHighlights val · QVHighlights val | mAP Avg.: 28.59 vs 32.30 | Hurt |
| add Dynamic Conv on top of self-attention (ablation) | Decoder with self-attention on QVHighlights val · QVHighlights val | mAP Avg.: 42.95 vs 41.08 without denoise; 44.56 vs 40.76 with denoise | Mixed |
| add boundary-denoising training (ablation) | Ego-style query-count study on MAD/TACoS-like setting with 8 queries · query-number ablation, epoch 10 | mAP/recall at epoch 10: 30.62 vs 26.30 at 10 epochs (8 queries) | Helped |
| increase query number from 8/16 to 32 with denoise training (ablation) | DenoiseLoc query-count study · query-number ablation, epoch 80 | accuracy at epoch 80: 44.11 at 32 queries vs 43.78 at 16 queries (denoise) | Mixed |
| add boundary-denoising training (ablation) | DenoiseLoc on THUMOS14 · THUMOS14 | mAP@0.5: 64.26 vs 62.62 | Helped |
| replace learned inference spans with random spans (ablation) | DenoiseLoc inference on QVHighlights · QVHighlights val | Avg. mAP (all): 44.51 vs 44.41 | No clear effect |
| use DDPM-style multi-step diffusion (DiffusionLoc) instead of DenoiseLoc (ablation) | DenoiseLoc denoising training · QVHighlights val | Avg. mAP: 44.26 at 1 step vs 44.66 DenoiseLoc; 42.99 at 512 steps | No clear effect |
| use DenoiseLoc with 30 queries instead of many-proposal methods | TACoS · TACoS R@5 | R@5 @0.5: 51.04 vs 56.56 (VLG-Net) | Hurt |
| use proposal-free DenoiseLoc with guidance model on MAD | MAD second-stage grounding · MAD | R@1 IoU=0.1 (with guidance): 11.59 vs 8.90 (CONE) | Helped |
| use DenoiseLoc on QVHighlights test | QVHighlights · QVHighlights test | mAP Avg.: 42.96 vs 36.12 (UMT) | Helped |
Surprising findings
- Applying a DDPM-style diffusion schedule (DiffusionLoc) gave no clear gain over the simpler denoising training.
- Random spans at inference perform competitively with learned spans (44.51 vs 44.41 avg mAP on Ego4D-style ablation).
Limitations
- Weaker R@5 on TACoS and Ego4D because the Hungarian matcher supervises only the top prediction.
- Performance sensitive to random seed in some examples; uncertainty varies across runs.
- Diffusion-style multi-step inference did not help and was impractical.
Experimental setup
Datasets: MAD QVHighlights TACoS THUMOS-14 Ego4D NLQ
Benchmarks: MAD QVHighlights TACoS THUMOS-14 Ego4D NLQ
Metrics: Recall@K at IoU thresholds mAP average mAP
Baselines: Moment-DETR VLG-Net Zero-shot CLIP CONE UMT XML XML+ MCN CAL VSLNet 2D-TAN ReLER
Builds on: Moment-DETR DN-DETR DiffusionDet DETR DynamicConv
Closest papers by meaning
- Learning Salient Boundary Feature for Anchor-free Temporal Action Localization (CVPR 2021 · similarity 0.89)
- Exploring Denoised Cross-Video Contrast for Weakly-Supervised Temporal Action Localization (CVPR 2022 · similarity 0.87)
- Masked Diffusion Vision-Language Models for Temporal Action Localization (NeurIPS 2026 · similarity 0.86)
- Boundary-Aware Temporal Dynamic Pseudo-Supervision Pairs Generation for Zero-Shot Natural Language Video Localization (AAAI 2025 · similarity 0.86)
- Energy vs. Noise: Towards Robust Temporal Action Localization in Open-World (AAAI 2025 · similarity 0.85)
- Rethinking Pseudo-Label Guided Learning for Weakly Supervised Temporal Action Localization from the Perspective of Noise Correction (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.