Research map / Video / Video generation

AVENUE: Audio-Video Editing Understanding and Evaluation

NeurIPS 2026 · Video › Video generation · approach: Video-to-audio synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces AVENUE, a human-verified audio-video editing benchmark (1,291 clips, 7,957 instructions over 12 edit types) with a sample-specific, modality-aware MLLM-judge evaluation, showing existing joint, sequential, and separate AV editors often leak edits into the unintended modality.

Problem

Audio-video editing requires inferring which modality should change and which must be preserved from the prompt, but existing benchmarks cover few edit types and modalities, and current evaluators are modality-blind and sample-agnostic.

Approach

A taxonomy-driven benchmark built from VGGSound with category-aware instruction templates, plus per-sample what_changed/what_preserved annotations that ground an MLLM judge (Qwen3-Omni) on four metrics, including Modality Selectivity, which checks whether the unintended modality stays intact.

Key results

• AvED (joint) achieves the highest Modality Selectivity on audio (97.5) but the lowest Edit Accuracy (64.1) among audio-targeted evaluations, indicating under-editing.
• RAVE + ZETA reaches Edit Accuracy 79.7 and audio Modality Selectivity 93.0, while RAVE + SDEdit scores 78.7 EA but only 51.4 MS.
• Qwen3-Omni judge reaches 67.0% pairwise agreement with humans versus a 71.1% human-human reference.

Surprising findings

• The joint model AvED scores highest on preservation but lowest on edit accuracy, suggesting high self-prompt preservation reflects under-editing rather than editing quality.
• The separated pipeline's audio-targeted edit advantage disappears on cross-modal temporal grounding (CMTG), where all models score only 20-29 out of 100.

Limitations

• Judge reliability is only moderate: Qwen3-Omni agreement with humans (67.0%) is below human-human agreement (71.1%), and some judges correlate weakly with humans.
• Category imbalance: Nature (14 clips) and Explosions (18 clips) are underrepresented relative to Instruments (481).
• Source clips are restricted to the VGGSound test set and filtered for person density, ROI consistency, and motion, which limits generality.

Experimental setup

Datasets: VGGSound

Benchmarks: AVENUE

Metrics: Edit Accuracy Modality Selectivity Perceptual Quality AV Consistency V-CLIP CLAP ImageBind DINO LPIPS Kendall tau-b

Baselines: RAVE + SDEdit RAVE + ZETA RAVE -> CAVE Source -> CAVE RAVE -> MMAudio AvED

Builds on: VGGSound RAVE MMAudio ZETA SDEdit

Closest papers by meaning

• SAO-Instruct: Free-form Audio Editing using Natural Language Instructions (NeurIPS 2025 · similarity 0.87)
• VE-Bench: Subjective-Aligned Benchmark Suite for Text-Driven Video Editing Quality Assessment (AAAI 2025 · similarity 0.87)
• Prompt-guided Precise Audio Editing with Diffusion Models (ICML 2024 · similarity 0.86)
• EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Models (AAAI 2025 · similarity 0.86)
• ThinkSound: Chain-of-Thought Reasoning in Multimodal LLMs for Audio Generation and Editing (NeurIPS 2025 · similarity 0.85)
• AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation (ICML 2026 · similarity 0.85)