Research map / Vision-language and multimodal / Multimodal LLMs

SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SpatialReward, a generative reward model for image editing that predicts edit-region boxes before reasoning (Think-with-Boxes), trained with SFT on a 260k spatial-aware dataset and GRPO, and used as the reward for online Flow-GRPO on OmniGen2.

Problem

Reward models for instruction-guided image editing evaluate edits poorly: pointwise MLLM judges neglect the source image ("Attention Collapse") and miss fine-grained cross-image inconsistencies, giving weak reward signals for online RL.

Approach

Anchor the judge's reasoning to explicitly predicted edit-region bounding boxes interleaved as tokens in the rationale, decompose scoring into semantic consistency and perceptual quality streams, and aggregate with a weighted geometric mean for dense rewards.

Key results

• SpatialReward-8B reaches 0.803 on EditReward-Bench vs 0.690 for EditScore-8B and 0.661 on MMRB2 overall (vs 0.570).
• In online Flow-GRPO on OmniGen2, SpatialReward improves GEdit-Bench VIEScore by +0.90 (6.42 to 7.32), vs +0.77 for EditReward and +0.45 for GPT-4.1 in the EditScore setting.
• On MER-Bench SpatialReward reaches 48.3% overall accuracy, close to Gemini-2.5-Pro (46.2%) and above EditScore-8B (35.0%).

Surprising findings

• Its generative VLM reward model had 1.5x lower per-image latency than the discriminative EditReward (72.7 vs 110.5 ms) due to vLLM prefix caching.
• Its MER-Bench 4-Pair accuracy (21.5%) exceeds Gemini-3.0-Flash (19.5%).

Limitations

• Relies on teacher annotations from proprietary models (GPT-5, Gemini) and a Qwen3-VL-235B pipeline for the 260k dataset; hallucination filtering is heuristic.
• Absolute gains on the stronger UniRef-Image-Edit backbone are small (+0.10 GEdit-Bench).
• MER-Bench overall accuracy remains low (48.3%), indicating complex multi-image judgment is still hard.

Experimental setup

Datasets: SPATIALREWARD-260K EditReward-Bench MMRB2 GEdit-Bench ImgEdit-Bench laion2B-en-aesthetic CC12M headshot pexels v1

Benchmarks: EditReward-Bench MMRB2 MER-Bench GEdit-Bench ImgEdit-Bench

Metrics: accuracy Kendall's tau VIEScore

Baselines: GPT-4.1 GPT-5 Gemini-2.5-Pro Gemini-3.0-Flash EditScore-8B EditReward Qwen3-VL-8B

Builds on: EditScore VIEScore Flow-GRPO GRPO EditReward

Closest papers by meaning

• SpatialFlow-GRPO: Where Spatial Credit Drives Image Editing (NeurIPS 2026 · similarity 0.90)
• SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.87)
• Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learning (CVPR 2026 · similarity 0.87)
• RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Reward (CVPR 2026 · similarity 0.87)
• Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing (NeurIPS 2026 · similarity 0.86)
• Exploring Spatial Intelligence from a Generative Perspective (CVPR 2026 · similarity 0.86)