Research map / Vision-language and multimodal / Multimodal LLMs
Decomposition of Concept-Level Rules in Visual Scenes
Fan Shi, Yuxuan Liang, Xiaolei Chen, Haiyang Yu, Xu Li, Yi Zheng, Rui Zhu, Xiangyang Xue, Bin Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CRD, a framework that decomposes visual scenes into independent concepts and the rules that change them, aiming to do so without hand-crafted inductive biases or human-designed priors.
Problem
Vision-language systems process images holistically with limited explicit decomposition of concepts and concept-changing rules; prior decomposition methods depend on hand-crafted inductive biases or human-designed priors.
Approach
A framework (CRD) that parses visual scenes into independent concepts and their concept-changing rules, learned without hand-crafted inductive biases or human-designed priors, in contrast to holistic processing and prior prior-dependent decomposition methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CRD (concept-rule decomposition with LVLM-MH sampling) to the LVLM | InternVL-3.5-8B · VSB-MA | Avg. Similarity: 59.9 -> 64.0 | Improved |
| add CRD to Qwen2.5-VL-7B | Qwen2.5-VL-7B · I-RAVEN | accuracy (%): 15.0 -> 89.3 | Improved |
| add CRD to Qwen2.5-VL-7B | Qwen2.5-VL-7B · RAVEN | accuracy (%): 59.7 -> 89.4 | Improved |
| add CRD to InternVL-3.5-8B | InternVL-3.5-8B · RAVEN / I-RAVEN | accuracy (%): RAVEN 11.6 -> 31.6 (InternVL3-8B vs InternVL-CRD) | Improved |
| remove the LVLM proposal ratio from the acceptance probability (ablation) | CRD (InternVL-3.5-8B) · VSB-MA | Avg. Similarity: 64.0 -> 61.0 | Hurt |
| remove the CRF score term from the acceptance probability (ablation) | CRD (InternVL-3.5-8B) · VSB-MA | Avg. Similarity: 64.0 -> 59.2 | Hurt |
| add patch-level scores on top of meta-attributes as prompt (ablation) | Qwen2.5-VL-7B + CRD · SpatialEval overall | overall accuracy: 61.45 -> 63.37 | Improved |
| add CRD-meta prompts to the LVLM | Qwen2.5-VL-7B · SpatialEval overall | overall accuracy: 60.11 -> 61.45 | Improved |
| add CRD-meta prompts to the LVLM | Qwen2.5-VL-3B · SpatialEval SpatialMap | accuracy: 50.93 -> 50.93 | No clear effect |
| add CRD at inference time | InternVL-3.5-8B · VSB-MA latency | latency per token (ms): 77.3 -> 234.3 | Hurt |
| increase patch grid from 2x2 to 4x4 (ablation) | CRD (InternVL-3.5-8B) · VSB-MA latency | latency per token (ms): 234.3 -> 239.5 | No clear effect |
| add CRD on human-annotated VSB-MA subset | InternVL-3.5-8B · VSB-MA (human-annotated) | Avg. Similarity: 58.56 -> 69.22 | Improved |
Closest papers by meaning
- Concept-RuleNet: Grounded Multi-Agent Neurosymbolic Reasoning in Vision Language Models (AAAI 2026 · similarity 0.90)
- VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs (CVPR 2026 · similarity 0.89)
- Visual Structures Help Visual Reasoning: Addressing the Binding Problem in LVLMs (NeurIPS 2025 · similarity 0.89)
- To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models (ICLR 2026 · similarity 0.89)
- Beyond Bag-of-Words: Diagnosing Compositional Binding Failures in Vision-Language Models (NeurIPS 2026 · similarity 0.89)
- What Makes a Maze Look Like a Maze? (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
