Research map / Large language models / Alignment and preferences
MM-RLHF: The Next Step Forward in Multimodal LLM Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
The provided text is the MME-RealWorld benchmark paper (ICML 2025 title is MM-RLHF, but the body describes a high-resolution, human-annotated evaluation benchmark), not an alignment dataset paper; the card reflects the body text.
Problem
Existing MLLM benchmarks have small data scale, model-generated annotations, and insufficient difficulty due to low image resolution, limiting assessment of real-world applicability.
Approach
A fully human-annotated benchmark of 29,429 QA pairs over 13,366 high-resolution images (avg ~2000x1500) across 5 real-world domains and 43 subtasks, plus a Chinese split (MME-RealWorld-CN).
Key results
- Even the best models fail to reach 60% accuracy on MME-RealWorld; InternVL-2 is the strongest perception model in the English split.
- Top reasoning model Claude 3.5 Sonnet reaches only 44.12% average accuracy on reasoning tasks; no top model exceeds 45%.
- LLaVA-1.5-7B achieves 24.9% on MME-RealWorld versus 58.2%+ on TextVQA-style benchmarks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a model designed for higher-resolution image input | MLLMs on MME-RealWorld · MME-RealWorld perception tasks | accuracy | Helped |
| use a model trained on large amounts of chart data | MLLMs on MME-RealWorld · Diagram and Table perception | accuracy | Helped |
| evaluate models on Chinese rather than English data | GPT-4o and GPT-4o-mini · MME-RealWorld-CN | accuracy: nearly 10% | Hurt |
Surprising findings
- Closed-source models often output option E (object not present) more than the actual frequency in the benchmark, suggesting conservative behavior or failure to perceive details.
- Several models perform much worse on the Chinese split for reasoning, e.g., GPT-4o drops nearly 10%.
Limitations
- Benchmark is multiple-choice, which may not capture open-ended generation quality.
- Closed-source models face image size limits and refusals that may confound results.
Experimental setup
Benchmarks: MME-RealWorld MME-RealWorld-CN
Metrics: accuracy average accuracy (Avg) class-based average accuracy (Avg-C)
Baselines: GPT-4o Gemini 1.5 Pro Claude 3.5 Sonnet InternVL-2 Qwen2-VL LLaVA1.5-7B
Builds on: MME MMBench SEED-Bench MMT-Bench
Closest papers by meaning
- RMB: Comprehensively benchmarking reward models in LLM alignment (ICLR 2025 · similarity 0.93)
- RLTHF: Targeted Human Feedback for LLM Alignment (ICML 2025 · similarity 0.91)
- Larger or Smaller Reward Margins to Select Preferences for LLM Alignment? (ICML 2025 · similarity 0.91)
- The Devil Is in the Details: Tackling Unimodal Spurious Correlations for Generalizable Multimodal Reward Models (ICML 2025 · similarity 0.91)
- BaseReward: A Strong Baseline for Multimodal Reward Model (ICLR 2026 · similarity 0.91)
- Generative RLHF-V: Learning Principles from Multi-modal Human Preference (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.