AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

P$^3$-VLM: A Point-based Alternative for Grounded 3D Vision-Language Models

Anna-Maria Halacheva, Jan-Nico Zaech, Sombit Dey, Luc V Gool, Danda Pani Paudel

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes P^3-VLM, a point-based alternative to bounding boxes for grounded 3D vision-language models, arguing box-based prompts and outputs leak semantic category information and let models ignore visual input.

Problem

Existing 3D VLM grounding benchmarks and prompting protocols rely heavily on bounding boxes, which leak object category semantics and create a shortcut where models neglect visual information during training.

Approach

Replace bounding-box spatial prompts and representation primitives with points, removing the semantic leakage that boxes carry, so that grounding must rely on visual evidence.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace box prompt with point promptblind LLM · Nr3Dclass accuracy (%): 36.4 -> 18.0Hurt
replace box prompt with point prompt (fine-tuned)LL3DA · Nr3Dclass accuracy (%): 47.7 -> 33.1Hurt
replace box prompt with point prompt (fine-tuned)GaussianVLM · Nr3Dclass accuracy (%): 46.2 -> 29.3Hurt
add the location-aware pathway (ablation)P3-VLM (p) · SQA3DEM-R: 45.6 -> 51.8Improved
add the context-aware pathway (ablation)P3-VLM (p) · ScanQAEM-R: 22.7 -> 31.9Improved
add the task-aware pathway (ablation)P3-VLM (p) · Nr3Dclass accuracy (%): 50.2 -> 52.1Improved
use full tri-pathway design instead of matched baseline (ablation)P3-VLM (p) · Nr3Dclass accuracy (%): 39.8 -> 52.1Improved
use raw voxel features instead of sparsified features for location pathway (ablation)P3-VLM (p) · Nr3Dclass accuracy (%): 42.3 -> 52.1Improved
use MLP instead of FiLM for location pathway (ablation)P3-VLM (p) · Nr3Dclass accuracy (%): 48.5 -> 52.1Improved
use 3D RoPE instead of learned absolute PE in sparsifier (ablation)P3-VLM (p) · Nr3Dclass accuracy (%): 49.4 -> 52.1Improved
train on random surface points instead of object centersP3-VLM (p) · Nr3Dclass accuracy (%): 52.1 -> 58.8Improved
point prompt instead of box prompt in trainingP3-VLM · GaussianVLM-OODEM: 15.42 -> 17.31Improved

Surprising findings

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy