Research map / Vision-language and multimodal / Multimodal LLMs
The Telephone Game: Evaluating Semantic Drift in Unified Models
Sabbir Mollah, Rohit Gupta, Swetha Sirnam, Qingyang Liu, Ahnaf Munir, Shah Mubarak
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| move from simple to complex compositional prompts (ablation) | BAGEL · GenEval-R MGG | MGG: 92.4 -> 75.9 | Hurt |
| move from simple to complex compositional prompts (ablation) | Janus-7B · GenEval-R MGG | MGG: 55.0 -> 14.9 | Hurt |
| move from simple to complex compositional prompts (ablation) | VILA-U · GenEval-R MGG | MGG: 49.5 -> 8.6 | Hurt |
| use BAGEL (shared-weight, large-scale interleaved pretraining) instead of VILA-U | unified model · Nocaps+Docci400 | MCD_avg: 0.2829 -> 0.4414 | Improved |
| replace CLIP cross-modal scorer with SigLIP2 (ablation) | SDP MCD · Nocaps+Docci400 | MCD_avg: 0.4414 -> 0.3592 | No clear effect |
| use MAGViT2 tokenizer features instead of CLIP features (ablation) | Show-o · Nocaps+Docci400 text-to-text MPNet | MCD: 0.5749 -> 0.5957 | Improved |
| use Detailed 1 recaption prompt instead of Short (ablation) | VILA-U · GenEval-R MGG | MGG: 19.66 -> 24.65 | Improved |
| use Detailed 1 recaption prompt instead of Short (ablation) | BAGEL · GenEval-R MGG | MGG: 78.14 -> 76.62 | No clear effect |
| use OWLv2 detector instead of GroundingDINO (ablation) | MGG · GenEval human audit | agreement: 24.9 -> 81.0 | Improved |
| evaluate multiple generations (MGG) instead of single-pass GenEval | evaluation protocol · human T2I ranking | Pearson correlation: -0.753 -> -0.821 | Improved |
| use proprietary ChatGPT instead of BAGEL | unified model · small subset | MGG: 83.3 -> 93.0 | Improved |
Closest papers by meaning
- RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmark (CVPR 2026 · similarity 0.87)
- ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.87)
- FineGRAIN: Evaluating Failure Modes of Text-to-Image Models with Vision Language Model Judges (NeurIPS 2025 · similarity 0.86)
- MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Models (ICLR 2026 · similarity 0.86)
- Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasoners (ICML 2026 · similarity 0.86)
- Evaluating Semantic Variation in Text-to-Image Synthesis: A Causal Perspective (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
