Research map / Vision-language and multimodal / Cross-modal learning
Perceptual Evaluation of Audio-Visual Synchrony Grounded in Viewers’ Opinion Scores
Lucas Goncalves, Prashant Mathur, Chandrashekhar Lavania, Metehan Cekic, Marcello Federico, Kyu Han
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add contrastive pre-training on audio-shift (stage 1) (ablation) | PEAVS · AVS benchmark test (set level) | Pearson correlation: 0.736 -> 0.794 | Improved |
| add contrastive pre-training on audio-shift (stage 1) (ablation) | PEAVS · AVS benchmark test (clip level) | Pearson correlation: 0.477 -> 0.536 | Improved |
| add cross-modal transformer layers instead of self-attention only (ablation) | PEAVS · AVS benchmark test (set level) | Pearson correlation: 0.720 -> 0.736 | Improved |
| use fused I3D+VGGish Frechet distance (FAVD) instead of unimodal FAD or FVD | Frechet-based AV synchrony metric · AVS benchmark test (set level) | Pearson correlation: 0.475 -> 0.527 over FAD; 0.321 -> 0.527 over FVD | Improved |
| use PEAVS instead of SparseSync for synchrony classification | AV shift detection · 400 videos, 21 bins | accuracy: 42.3% -> 50.5% | Improved |
| PEAVS vs SparseSync on detecting distorted content | AV shift detection · 400 videos, 21 bins | distortion detection: SparseSync 18% edge in detecting distortions | Hurt |
Closest papers by meaning
- FAVE: A Structured Benchmark for Fine-Grained Audio-Visual Temporal Evaluation in Multimodal LLMs (CVPR 2026 · similarity 0.87)
- JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation (ICLR 2026 · similarity 0.87)
- $\texttt{AVROBUSTBENCH}$: Benchmarking the Robustness of Audio-Visual Recognition Models at Test-Time (NeurIPS 2025 · similarity 0.87)
- Adapting to the Unknown: Training-Free Audio-Visual Event Perception with Dynamic Thresholds (CVPR 2025 · similarity 0.86)
- SVHalluc: Benchmarking Speech-Vision Hallucination in Audio-Visual Large Language Models (CVPR 2026 · similarity 0.86)
- DiVAS: Video and Audio Synchronization with Dynamic Frame Rates (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
