Research map / Vision-language and multimodal / Multimodal LLMs
PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PeRL, a GRPO-based RL method for interleaved multi-image reasoning that permutes image orders (with text rephrasing) and filters rollouts by difficulty, training Qwen2.5-VL-7B to state-of-the-art on multi-image benchmarks while staying competitive on single-image math.
Problem
Vision-language models lack robust reasoning across multiple interleaved images, showing positional bias when image order changes, and RL-trained multimodal reasoners trained on single-image data fail to generalize to multi-image positional reasoning.
Approach
Augments GRPO training with random permutations of image sequences (with semantically adjusted answers) to expose positional diversity, computes advantages jointly across original and permuted groups, and resamples training questions by rollout-based difficulty so that learning focuses on informative samples.
Key results
- Qwen-PeRL reaches 76.39 on Mantis-Eval and 58.53 on BLINK, the best among compared models on these benchmarks.
- Average across 8 metrics is 51.13, versus 50.05 for the next best model (Noisy-K12) and 47.90 for the Qwen2.5-VL-7B base.
- Ablation: n_s=1 permutation improves average over no permutation; n_s=2 gives higher Mantis-Eval (77.78) but lower MathVista and Remi.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PeRL instead of the strongest reported baseline | Qwen2.5-VL-7B-Instruct · Mantis-Eval | score: +5.59 | Helped |
| Use PeRL instead of the strongest reported baseline | Qwen2.5-VL-7B-Instruct · BLINK | score: +3.30 | Helped |
| Use PeRL instead of the strongest reported baseline | Qwen2.5-VL-7B-Instruct · MMIU | score: +2.23 | Helped |
| Use PeRL instead of the strongest reported baseline | Qwen2.5-VL-7B-Instruct · MathVista | score: +4.80 | Helped |
| Use PeRL instead of the strongest reported baseline | Qwen2.5-VL-7B-Instruct · MathVerse | score: +3.26 | Helped |
| Use PeRL instead of the strongest reported baseline | Qwen2.5-VL-7B-Instruct · MathVision | score: +3.19 | Helped |
| Use PeRL instead of the strongest reported baseline | Qwen2.5-VL-7B-Instruct · Remi | score: +7.00 | Helped |
| Use PeRL instead of the strongest reported baseline | Qwen2.5-VL-7B-Instruct · MV-MATH | score: +5.27 | Helped |
| Add one permuted sample rather than no permutation (ablation) | Permutation GRPO · Mantis-Eval | score: 75.93 -> 76.39 | Helped |
| Increase permutation count from one to two samples (ablation) | Permutation GRPO · BLINK | score: 58.53 -> 59.50 | Helped |
| Increase permutation count from one to two samples (ablation) | Permutation GRPO · MathVista | score: 73.0 -> 72.2 | Hurt |
| Increase permutation count from one to three samples (ablation) | Permutation GRPO · Mantis-Eval | score: 76.39 -> 74.07 | Hurt |
Surprising findings
- Increasing permutation count to n_s=2 degrades reasoning-centric benchmarks such as MathVista and Remi while improving spatial benchmarks.
- Training only on single-image data reduces multi-image performance, while multi-image training alone hurts single-image reasoning.
Limitations
- Permutation with n_s=2 or more harms reasoning-centric benchmarks and increases training cost.
- Semantic equivalence and answer remapping after permutation rely on GPT-4o prompting, which may be noisy.
- Evaluation on several benchmarks uses vLLM/VLMEvalKit with differing decoding setups and some baseline numbers are copied from prior work.
Experimental setup
Datasets: Mantis-Instruct K12
Benchmarks: Mantis-Eval BLINK MMIU MathVista MathVerse MathVision Remi MV-MATH
Metrics: accuracy
Baselines: Qwen2.5-VL-7B Qwen2-VL-7B Migician-630K LLaVA-OneVision InternVL2.5 R1-VL-7B-260K R1-OneVision-7B MM-Eureka Noisy-K12 GPT-4o
Builds on: GRPO DeepSeek-R1 NoisyRollout Mantis-Instruct veRL
Closest papers by meaning
- Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.91)
- More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.91)
- Unveiling Chain of Step Reasoning for Vision-Language Models with Fine-grained Rewards (NeurIPS 2025 · similarity 0.91)
- Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models (NeurIPS 2025 · similarity 0.91)
- Improving Vision-language Models with Perception-centric Process Reward Models (CVPR 2026 · similarity 0.90)
- Perception-Aware Policy Optimization for Multimodal Reasoning (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.