Research map / Vision-language and multimodal / Multimodal LLMs
They Can See It but not Say It: Iterative Self Knowledge Re-expression in Visual Reasoning Relative Pose Identification
MENGYU WANG, Ken Deng, Yftah Ziser, Shay Cohen, Tiejun Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes stable output biases of vision-language models on visual reasoning relative pose identification (VRRPI) and proposes an iterative self-knowledge re-expression approach to mitigate them.
Problem
Modern VLMs show weak multi-view 3D spatial reasoning on visual reasoning relative pose identification, where they disproportionately predict specific motion patterns regardless of visual evidence.
Approach
Identifies stable output biases in VLMs on VRRPI and proposes an iterative self-knowledge re-expression procedure to mitigate them; the abstract is truncated so the exact mechanism is not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply SKR-ID (classification head + iterative median-balanced self-annotation) | LLaVA-OneVision · VRRPI-Bench | Macro-F1: 35.0 -> 58.0 | Improved |
| apply SKR-ID | Qwen3-VL · VRRPI-Bench | Macro-F1: 39.5 -> 53.8 | Improved |
| apply SKR-ID | VST-RL · VRRPI-Bench | Macro-F1: 45.4 -> 53.7 | Improved |
| add object-centric debiasing prompt for self-annotation (ablation) | LLaVA-OneVision · VRRPI-Bench | Macro-F1: 35.0 -> 49.1 | Improved |
| add one round of SKR with classification head (ablation) | LLaVA-OneVision · VRRPI-Bench | Macro-F1: 49.1 -> 53.2 | Improved |
| add median-threshold balancing during iteration (ablation) | LLaVA-OneVision · VRRPI-Bench | Macro-F1: 54.0 -> 58.0 | Improved |
| remove object-centric prompt at initialization (ablation) | LLaVA-OneVision · VRRPI-Bench | Macro-F1: 58.0 -> 33.3 | Hurt |
| initialize with random balanced labels instead of prompted pseudo-labels (ablation) | LLaVA-OneVision · VRRPI-Bench | Macro-F1: 58.0 -> 50.8 | Hurt |
| use 0.7-quantile threshold instead of median (ablation) | LLaVA-OneVision · VRRPI-Bench | Macro-F1: 58.0 -> 47.2 | Hurt |
| train on 70/30 pseudo-label subset instead of balanced (ablation) | LLaVA-OneVision · VRRPI-Bench | Macro-F1: 58.0 -> 56.7 | Hurt |
| median thresholding on frozen VLM scores only (no head or fine-tuning) (ablation) | LLaVA-OneVision · VRRPI-Bench | Macro-F1: 35.0 -> 32.3 / 34.3 / 43.4 | Mixed |
| use self-consistency over 5 prompted responses (ablation) | LLaVA-OneVision · VRRPI-Bench | Macro-F1: 49.1 -> 49.8 | No clear effect |
Surprising findings
- VLMs disproportionately predict specific motion patterns regardless of visual evidence, i.e., stable output biases.
Closest papers by meaning
- Expand VSR Benchmark for VLLM to Expertize in Spatial Rules (AAAI 2025 · similarity 0.89)
- SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs (ICLR 2026 · similarity 0.88)
- InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.88)
- ReVSI: Rebuilding Visual Spatial Intelligence Evaluation for Accurate Assessment of VLM 3D Reasoning (ICML 2026 · similarity 0.87)
- ConsisVLA-4D: Advancing Spatiotemporal Consistency in Efficient 3D-Perception and 4D-Reasoning for Robotic Manipulation (CVPR 2026 · similarity 0.87)
- Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
