Research map / Video / Video generation
PAVAS: Physics-Aware Video-to-Audio Synthesis
Oh Hyun-Bin, Yuhta Takida, Toshimitsu Uesaka, Tae-Hyun Oh, Yuki Mitsufuji
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add physics conditioning (mass and velocity) via Phy-Adapter | MMAudio-L backbone · VGGSound test | FD PaSST: 60.60 -> 47.38 | Improved |
| add physics conditioning (mass and velocity) | MMAudio-L backbone · VGG-Impact | APCC-delta: 0.536 -> 0.378 | Improved |
| add physics conditioning | MMAudio-L backbone · VGGSound test | DeSync: 0.442 -> 0.446 | No clear effect |
| add physics conditioning | MMAudio-L backbone · VGGSound test | FD VGG: 0.97 -> 1.15 | Hurt |
| train the backbone longer on VGGSound without physics (ablation) | backbone (S-16kHz) · VGGSound | FD PaSST: 70.19 -> 71.99 | No clear effect |
| add mass conditioning only (ablation) | backbone (S-16kHz) · VGGSound | FD PaSST: 70.19 -> 66.89 | Improved |
| add velocity conditioning only (ablation) | backbone (S-16kHz) · VGGSound | DeSync: 0.483 -> 0.446 | Improved |
| add both mass and velocity conditioning (ablation) | backbone (S-16kHz) · VGGSound | FD PaSST: 70.19 -> 65.67 | Improved |
| use delta-modulation instead of direct summation (ablation) | Phy-Adapter · VGGSound | FD PaSST: 67.31 -> 65.67 | Improved |
| add physics conditioning | MMAudio-L backbone · User study | physical plausibility: 3.90 -> 4.37 | Improved |
Closest papers by meaning
- Benchmarking Single-Factor Physical Video-to-Audio Generation (CVPR 2026 · similarity 0.90)
- Physics-Driven Diffusion Models for Impact Sound Synthesis From Videos (CVPR 2023 · similarity 0.90)
- AV-GS: Learning Material and Geometry Aware Priors for Novel View Acoustic Synthesis (NeurIPS 2024 · similarity 0.87)
- Tri-Ergon: Fine-Grained Video-to-Audio Generation with Multi-Modal Conditions and LUFS Control (AAAI 2025 · similarity 0.86)
- PrismAudio: Decomposed Chain-of-Thought and Multi-dimensional Rewards for Video-to-Audio Generation (ICLR 2026 · similarity 0.85)
- SonicVisionLM: Playing Sound with Vision Language Models (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
