Research map / Vision-language and multimodal / Multimodal LLMs
Seeing Isn't Orienting: A Cognitively Grounded Hierarchical Benchmark for Object Orientation in MLLMs
Nazia Tasnim, Keanu Nichols, Yuting Yan, Nicholas Ikechukwu, Elva Zou, Deepti Ghadiyaram, Bryan Plummer
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use token-based fusion instead of linear projection (ablation) | Qwen2.5-3B-Inst. LoRA · DORI (LoRA fusion comparison) | average coarse accuracy (%): 41.2 -> 74.3 | Improved |
| use token-based fusion instead of linear projection (ablation) | Qwen2.5-3B-Inst. LoRA · DORI (LoRA fusion comparison) | average granular accuracy (%): 19.1 -> 46.6 | Improved |
| add bottleneck to linear projection (ablation) | Qwen2.5-3B-Inst. LoRA · DORI (LoRA fusion comparison) | average coarse accuracy (%): 41.2 -> 32.1 | Hurt |
| LoRA finetune on DORI | Qwen2.5-3B-Inst. · DORI held-out split (LoRA finetuning) | average coarse accuracy (%): 39.6 -> 79.5 | Improved |
| LoRA finetune on DORI | Qwen2.5-3B-Inst. · DORI held-out split (LoRA finetuning) | average granular accuracy (%): 16.6 -> 50.2 | Improved |
| LoRA finetune on DORI | Qwen2.5-VL-3B · SAT (zero-shot) | accuracy (%): 51.3 -> 63.3 | Improved |
| LoRA finetune on DORI | Qwen2.5-VL-3B · Blink Visual-Correspondence (zero-shot) | accuracy (%): 25.3 -> 26.5 | No clear effect |
| add structured prompt (task description, examples, steps) (ablation) | LLaVA-Next-8B · DORI View Parallelism granular | accuracy: 93% relative decline without structure | Improved |
| add structured prompt (ablation) | Mantis-Idfs-8B · DORI Compound Rotation coarse | accuracy: 141% relative surge without structure | Hurt |
| add structured prompt (ablation) | DS-7B-Chat · DORI Inter-Object Direction coarse and Compound Rotation coarse | accuracy: 145% relative gain and 97% collapse without structure | Mixed |
| use instruction-tuned DeepSeek-7B-Chat instead of Base | DeepSeek-7B · DORI (full) | average coarse accuracy (%): 23.1 -> 33.9 | Improved |
| upgrade Qwen2.5-3B to Qwen3-8B | Qwen family · DORI (full) | average coarse accuracy (%): 35.0 -> 49.3 | Improved |
Closest papers by meaning
- SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs (ICLR 2026 · similarity 0.92)
- SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation (NeurIPS 2025 · similarity 0.90)
- 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models (ICML 2026 · similarity 0.90)
- Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs (AAAI 2026 · similarity 0.89)
- BOP-ASK: Object-Interaction Reasoning for Vision-Language Models (CVPR 2026 · similarity 0.89)
- Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
