Research map / Vision-language and multimodal / Multimodal LLMs

ReasonX: MLLM-Guided Intrinsic Image Decomposition

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Uses a fine-tuned MLLM judge (InternVL2.5-4B) giving relative intrinsic comparisons as GRPO rewards to fine-tune PRISM and Marigold IID models on unlabeled COCO images, improving zero-shot albedo, depth and normals.

Problem

Intrinsic image decomposition models trained on synthetic paired data generalize poorly to diverse real-world images, and absolute ground-truth intrinsics are costly to obtain.

Approach

Treat an MLLM's pairwise relative judgments about intrinsic properties (depth order, normal front-facingness, shading, material similarity) as rewards; compare them with analytic relations derived from the predicted intrinsic maps, and optimize the base diffusion/flow model with Flow-GRPO on real unlabeled images.

Key results

• PRISM-X reduces IIW WHDR 10% from 17.2 to 12.9 (+25.0%) and Marigold-X from 16.7 to 15.2 (+9.0%).
• PRISM-X improves ETH3D AbsRel from 0.142 to 0.077 (+45.8%) in zero-shot depth.
• PRISM-X achieves 15.7° mean normal error on NYUv2 vs 16.1° for PRISM, and 14.5° on DIODE vs 14.6°.

Surprising findings

• Derived relative judgments from an absolute 5-bin MLLM classifier perform much worse than a directly trained relative judge (e.g., depth 47.5% vs 96.2%).
• Removing KL regularization degrades results consistently, indicating KL is needed for stable comparative-reward optimization.

Limitations

• Relies on point-pair sampling and modality-wise rewards, which limits joint multi-modal signals.
• Judge accuracy is lower for albedo (89.4%) and irradiance (87.6%) than for depth.
• Normal estimation gains are modest, and PRISM-X underperforms on DIODE 11.25° accuracy.

Experimental setup

Datasets: COCO HyperSim InteriorVerse

Benchmarks: IIW MAW NYUv2 ETH3D DIODE HyperSim InteriorVerse ARAP

Metrics: WHDR intensity chromaticity AbsRel delta1 mean angular error 11.25° accuracy PSNR LPIPS RMSE

Baselines: Marigold Depth v1.0 Marigold-Normals v1.1 Depth Anything V2 DSINE GeoWizard StableNormal OmniGen2 CRefNet

Builds on: GRPO Flow-GRPO Marigold PRISM

Closest papers by meaning

• Perception Tokens Enhance Visual Reasoning in Multimodal Language Models (CVPR 2025 · similarity 0.85)
• IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Prompting (CVPR 2026 · similarity 0.84)
• Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models (CVPR 2025 · similarity 0.84)
• OpenView: Empowering MLLMs with Out-of-view VQA (NeurIPS 2026 · similarity 0.84)
• Intrinsic Image Diffusion for Indoor Single-view Material Estimation (CVPR 2024 · similarity 0.84)
• ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning (CVPR 2025 · similarity 0.84)