Research map / Vision-language and multimodal / Multimodal LLMs

Let's Think in Two Steps: Mitigating Agreement Bias in MLLMs with Self-Grounded Verification

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Identifies "agreement bias" in MLLM verifiers (over-validation of flawed agent trajectories) across web, desktop, and robotics, and proposes Self-Grounded Verification (SGV), a zero-shot two-step method that first elicits task priors and then judges the trajectory conditioned on them.

Problem

MLLMs used as verifiers/reward models for open-ended agent tasks (web, computer use, robotics) tend to over-validate flawed behavior, yielding low true-negative rates and harming self-improvement, online supervision, and evaluation.

Approach

Split verification into two steps: first prompt the MLLM to generate general priors about how the task is accomplished, conditioned only on the task and initial screenshot; then have it evaluate the full trajectory conditioned on those self-generated priors. This modulates conditional generation to better use the model's knowledge and reduce over-validation.

Key results

• SGV improves TNR by up to 25 pp and accuracy by up to 14 pp over no-SGV verifiers across models and benchmarks (Table 1).
• In online supervision, SGV gives +9 pp success rate on VisualWebArena (20% relative) and +5 pp on OSWorld (22% relative), surpassing the previous VWA state of the art by 20 pp.
• Self-improvement with SGV verifier boosts success rate by up to 10.4 pp on VisualWebArena, where the baseline verifier plateaus.

Surprising findings

• Agreement bias persists across 28+ scoring templates, test-time scaling (CoT, majority voting, thinking models), and across verifiers and agents from different families.
• Binary success/failure scales amplify over-validation, and a verifier with over 90% recall can still misclassify around 50% of failed trajectories.

Limitations

• SGV reduces but does not eliminate agreement bias.
• Remaining failures often stem from base MLLMs' weak integration of visual perception and language.
• Relies on benchmark oracles as proxies for human judgment, and VWA oracles were manually corrected.

Experimental setup

Datasets: VisualWebArena trajectories OSWorld trajectories robomimic tool-hang demonstrations AgentRewardBench

Benchmarks: VisualWebArena OSWorld robomimic AgentRewardBench

Metrics: accuracy TPR TNR bias distance skewness success rate

Baselines: No SGV (binary) CoT majority voting rubric-based verifier (AgentRewardBench approach) Thinking models VisualWebArena oracle

Builds on: chain-of-thought set-of-marks prompting Reflexion ReAct AgentRewardBench

Closest papers by meaning

• MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark (ICML 2024 · similarity 0.88)
• LLM Agents Can Be Choice-Supportive Biased Evaluators: An Empirical Study (AAAI 2025 · similarity 0.88)
• Enhancing the Outcome Reward-based RL Training of MLLMs with Self-Consistency Sampling (NeurIPS 2025 · similarity 0.87)
• MLRC-Bench: Can Language Agents Solve Machine Learning Research Challenges? (NeurIPS 2025 · similarity 0.87)
• Do as We Do, Not as You Think: the Conformity of Large Language Models (ICLR 2025 · similarity 0.87)
• VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environments (CVPR 2026 · similarity 0.87)