Research map / Vision-language and multimodal / Multimodal LLMs
GAP-MLLM: Geometry-Aligned Pre-training for Activating 3D Spatial Perception in Multimodal Large Language Models
Jiaxin Zhang, Junjun Jiang, Haijie LI, Youyu Chen, Kui Jiang, Dave Zhenyu Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add sparse joint geometry-semantics pre-training (ablation) | GAP-MLLM-3B · 3D video detection | F1: 44.7 -> 48.7 | Improved |
| add multi-level gated fusion (ablation) | GAP-MLLM-3B · 3D video detection | F1: 44.7 -> 47.5 | Improved |
| add both pre-training and gated fusion (ablation) | GAP-MLLM-3B · 3D video detection | F1: 44.7 -> 50.6 | Improved |
| add GAP to VG-LLM (ablation) | VG-LLM-4B · 3D video detection | F1: 38.2 -> 43.5 | Improved |
| add joint pre-training to pure Qwen3-VL (ablation) | Qwen3-VL · 3D video detection | F1: 39.7 -> 42.4 | Improved |
| add semantic labels to pointmap-only pre-training (ablation) | VG-LLM · 3D video detection | F1: 40.1 -> 41.4 | Improved |
| add pointmap pre-training (ablation) | VG-LLM · 3D video detection | F1: 38.2 -> 40.1 | Improved |
| add two-stage grounding (ablation) | VG-LLM · ScanRefer | Acc@0.25: 37.5 -> 45.4 | Improved |
| add sparse joint pre-training (ablation) | GAP-MLLM-3B · ScanRefer | Acc@0.25: 47.9 -> 51.7 | Improved |
| use gated fusion over addition, weighted fusion and cross-attention (ablation) | GAP-MLLM · fusion strategy ablation | accuracy | Improved |
| blur first frame (sigma=12) (ablation) | GAP-MLLM · 3D video detection | F1: 50.6 -> 46.0 | Hurt |
| occlude 30% of first frame (ablation) | GAP-MLLM · 3D video detection | F1: 50.6 -> 50.1 | No clear effect |
Closest papers by meaning
- Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence (NeurIPS 2025 · similarity 0.92)
- Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models (NeurIPS 2025 · similarity 0.91)
- Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning (CVPR 2026 · similarity 0.91)
- Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering (NeurIPS 2026 · similarity 0.91)
- MLLMs Need 3D-Aware Representation Supervision for Scene Understanding (NeurIPS 2025 · similarity 0.91)
- MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
