Research map / Vision-language and multimodal / Multimodal LLMs

OddGridBench: Exposing the Lack of Fine-Grained Visual Discrepancy Sensitivity in Multimodal Large Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces OddGridBench, a controllable odd-one-out grid benchmark for MLLM visual discrepancy sensitivity, and OddGrid-GRPO, an RL method (curriculum + distance-aware reward) that lifts Qwen3-VL-2B from 17.14% to 82.64%.

Problem

MLLMs are rarely evaluated on low-level fine-grained visual discrepancy detection (small differences in color, size, rotation, or position among visually similar items), and existing odd-one-out frameworks are not suited to MLLMs.

Approach

A parameterized generator of icon grids with one odd element differing by controlled color (CIE-Lab dE), size, rotation, or position amounts, plus an RL method that combines curriculum ordering by difficulty with a Gaussian distance-aware reward on row-column predictions instead of binary exact-match reward.

Key results

• Qwen3-VL-2B with OddGrid-GRPO reaches 82.64% total accuracy on OddGridBench, versus 17.14% for the base model and 70.86% for standard GRPO.
• Best evaluated model Qwen3-VL-32B reaches 68.07% total accuracy, while human annotators reach 87.47%.
• Gemini-2.5-Pro reaches 49.29% and GPT-5 reaches 28.93% total accuracy on OddGridBench.

Surprising findings

• Qwen3-VL-2B and 4B outperform much larger models such as InternVL3.5-38B and Molmo-72B on OddGridBench.
• Proprietary systems show no clear advantage over leading open-source models on this task.

Limitations

• Synthetic icon grids may not transfer fully to real-world images; cross-dataset results are reported only in supplementary material.
• Evaluation relies on a fixed prompt and row-column answer format; bounding-box grounding was found to perform poorly.
• Human evaluation uses only 350 sampled instances with three annotators.

Experimental setup

Datasets: MVTec-AD VisA MNIST Similar Chinese Characters

Benchmarks: OddGridBench MVTec-AD VisA MNIST SCC

Metrics: accuracy tolerance accuracy labeled grid accuracy EM F1 IoU@0.3/0.4/0.5

Baselines: GRPO GSPO Qwen3-VL-2B (base) Gemini-2.5-Pro GPT-5 InternVL3.5-38B Qwen3-VL-32B

Builds on: GRPO GSPO EasyR1

Closest papers by meaning

• Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment (CVPR 2025 · similarity 0.90)
• CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.90)
• BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception (ICML 2026 · similarity 0.89)
• Predictive Regularization Against Visual Representation Degradation in Multimodal Large Language Models (CVPR 2026 · similarity 0.89)
• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception (ICML 2026 · similarity 0.89)
• DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Models (CVPR 2026 · similarity 0.89)