Research map / Vision-language and multimodal / Multimodal LLMs

Visual Jigsaw Post-Training Improves MLLMs

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Visual instruction tuning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Visual Jigsaw, a self-supervised RLVR (GRPO) post-training task where MLLMs reorder shuffled image patches, video clips, or depth-sorted 3D points, improving vision-centric perception of Qwen2.5-VL-7B without architectural changes.

Problem

Current MLLM RL post-training is text-centric, leaving intrinsic visual understanding (fine-grained perception, temporal and 3D spatial understanding) underdeveloped, and existing visual-reconstruction approaches need extra generative modules.

Approach

Formulate vision-centric self-supervision as a verifiable ordering (jigsaw) problem answered in natural language, optimized with GRPO using a graded partial-credit permutation reward, across images, videos and RGB-D 3D data.

Key results

• Image Jigsaw on Qwen2.5-VL-7B gains +6.00 on MMVP and +6.06 on fine-grained MMStar subset over the base model, with a +1.43 average-style gain on the reported image suite.
• Video Jigsaw (64 frames) improves AoTBench from 52.41 to 57.64 and CVBench from 51.50 to 54.50 over Qwen2.5-VL-7B.
• 3D Jigsaw raises DA-2K from 54.45 to 71.56 and SAT-Real from 48.66 to 64.00 over Qwen2.5-VL-7B.

Surprising findings

• SFT on the image jigsaw task yields smaller gains and degrades some benchmarks (e.g., LISA-Grounding, OVD-Eval), whereas RL generalizes better.
• Enabling chain-of-thought can degrade some reasoning MLLMs on vision-centric benchmarks (e.g., OVD-Eval 35.78 to 31.44 for ThinkLite-VL).

Limitations

• Image and video jigsaw use relatively simple standard formulations; richer configurations unexplored.
• Scaling of data and model size was limited by compute; scalability unknown.
• Some 3D jigsaw variants (view-motion, BEV-pose matching) did not improve downstream results.

Experimental setup

Datasets: COCO LLaVA-Video ScanNet

Benchmarks: MMVP MMStar MMBench HR-Bench V* MME-RealWorld LISA-Grounding OVD-Eval VSR OmniSpatial DA-2K Winoground

Metrics: accuracy

Baselines: ThinkLite-VL VL-Cogito LLaVA-Critic-R1 Video-R1 Jigsaw-R1 Image Jigsaw (SFT)

Builds on: GRPO RLVR Jigsaw-R1 Video-R1

Closest papers by meaning

• Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.92)
• From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.90)
• Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.90)
• Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.90)
• PGT: Procedurally Generated Tasks for improving visual grounding in MLLMs (ICML 2026 · similarity 0.90)
• Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.89)