Research map / Vision-language and multimodal / Multimodal LLMs
Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models
Yijie Qian, Juncheng Wang, Chao Xu, Huihan Wang, Yuxiang Feng, Yang Liu, Baigui Sun, Yong Liu, Shujun Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace regression with scalar ranking (BTL) head (ablation) | Omni-LLM evaluator · SynthSync test | Kendall tau: 0.1628 -> 0.4515 | Improved |
| score as generated text instead of continuous scalar head (ablation) | Omni-LLM evaluator · SynthSync test | Pair acc: 71.07 -> 56.19 | Hurt |
| train on SynthSync instead of zero-shot Qwen2.5-Omni-3B (ablation) | Omni-LLM evaluator · SynthSync test | Pair acc: 51.02 -> 66.60 | Improved |
| add preference alignment fine-tuning (PFT) (ablation) | Omni-LLM evaluator · SynthSync test | Pair acc: 66.60 -> 71.16 | Improved |
| add R-GRPO reinforcement post-training (ablation) | Omni-LLM evaluator · SynthSync test | Pair acc: 71.16 -> 72.38 | Improved |
| use listwise reward only in R-GRPO (ablation) | R-GRPO · SynthSync test | Kendall tau: 0.4515 -> 0.4267 | Hurt |
| use pairwise reward only in R-GRPO (ablation) | R-GRPO · SynthSync test | Kendall tau: 0.4515 -> 0.4700 | Improved |
| combine pairwise and listwise rewards (dual) (ablation) | R-GRPO · SynthSync test | Kendall tau: 0.4515 -> 0.4899 | Improved |
| set exploration sigma to 10.0 instead of 2.5 (ablation) | R-GRPO · SynthSync test | Pair acc: 72.38 -> 71.70 | Hurt |
| set exploration sigma to 2.0 instead of 2.5 (ablation) | R-GRPO · SynthSync test | Pair acc: 72.38 -> 71.56 | Hurt |
| use the learned metric as Best-of-N reward instead of AV-Align | LTX-2 BoN · LTX-2 BoN, AV-Align score | score: 0.0698 -> 0.1439 (Random vs +Ours) | Improved |
| use VQ-Insight instead of our evaluator for bottom-1 detection | Omni-LLM evaluator · SynthSync test | Bottom-1 acc: VQ-Insight 62.80 vs Ours 55.00 | Mixed |
Closest papers by meaning
- FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance (NeurIPS 2026 · similarity 0.86)
- UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalities (NeurIPS 2026 · similarity 0.85)
- JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation (ICLR 2026 · similarity 0.85)
- Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners (CVPR 2024 · similarity 0.85)
- V2A-Mapper: A Lightweight Solution for Vision-to-Audio Generation by Connecting Foundation Models (AAAI 2024 · similarity 0.85)
- XModBench: Benchmarking Cross-Modal Capabilities and Consistency in Omni-Language Models (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
