Research map / Video / Video generation
Conditional Flow Matching for Visually-Guided Acoustic Highlighting
Hugo Malard, Gael Le Lan, Daniel Wong, David Alon, YI-CHIAO WU, Sanjeel Parekh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use flow matching with CLIP-CLAP conditioning instead of discriminative VisAH | VisAH · Muddy Mix test | KLD: 11.28 -> 9.70 | Improved |
| add CLIP-CLAP adapter conditioning to discriminative VisAH (ablation) | VisAH · Muddy Mix test | KLD: 11.37 -> 11.28 | Improved |
| add rollout loss to flow matching (ablation) | VisAH-FM · Muddy Mix test | KLD: 10.99 -> 9.79 | Improved |
| add rollout loss to flow matching (ablation) | VisAH-FM · Muddy Mix test | LDif: 9.48 -> 7.87 | Improved |
| add consistency loss to flow matching (ablation) | FM · Muddy Mix test | KLD: 10.99 -> 14.58 | Hurt |
| inject Gaussian noise (bridge matching) into trajectories (ablation) | FM · Muddy Mix test | KLD: 10.99 -> 10.85 | Improved |
| weight FM loss inversely to time (ablation) | FM · Muddy Mix test | KLD: 10.99 -> 11.16 | Hurt |
| add warm start (source sample in vector field) (ablation) | FM only · Muddy Mix test | KLD: 16.52 -> 10.99 | Improved |
| add audio-adapter conditioning (V+A) over vision only (ablation) | VisAH-FM · Muddy Mix test | KLD: 9.79 -> 9.70 | Improved |
| add text to vision-audio adapter conditioning (ablation) | VisAH-FM · Muddy Mix test | KLD: 9.70 -> 9.73 | No clear effect |
| vary rollout weight lambda 0.1 to 1.0 (ablation) | VisAH-FM · Muddy Mix test | KLD: 9.85, 9.79, 9.93, 9.82 | No clear effect |
| increase inference steps from 4 to 10 (ablation) | VisAH-FM · Muddy Mix test | LDif: 7.87 -> 8.35 | Hurt |
Closest papers by meaning
- Learning to Highlight Audio by Watching Movies (CVPR 2025 · similarity 0.88)
- Foley-Flow: Coordinated Video-to-Audio Generation with Masked Audio-Visual Alignment and Dynamic Conditional Flows (CVPR 2025 · similarity 0.88)
- Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes (CVPR 2025 · similarity 0.87)
- UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalities (NeurIPS 2026 · similarity 0.86)
- Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners (CVPR 2024 · similarity 0.86)
- CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
