Research map / Point clouds / Point cloud classification
LIBA: Language Instructed Multi-granularity Bridge Assistant for 3D Visual Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LIBA, a 3D visual grounding framework that bridges point-cloud and text encoders at multiple granularities (Dynamic Bridge Adapter, Language-aware Cross-scale Object Modulation) and uses LLM-guided Attribute-then-Relation query selection, reaching state-of-the-art on ScanRefer and Nr3D/Sr3D.
Problem
3D visual grounding suffers from granularity inconsistency between point cloud features and text features, since fusion happens at a single scale and language-agnostic multi-scale fusion worsens the mismatch; existing methods also lack LLM world knowledge and interpretable reasoning for ambiguous re
Approach
Bilateral Dynamic Bridge Adapters interleave point-cloud (PointNet++) and text (DeBERTa) encoder blocks for top-down cross-modal alignment; language-modulated cross-scale fusion of point features; and an LLM (LoRA-tuned, with a <GROUND> token) generates attribute-aware queries, refined by a scene-graph relation step, in a hierarchical Attribute-then-Relation query selection.
Key results
- On ScanRefer, LIBA reaches 59.57% overall Acc@0.25 and 48.96% Acc@0.5, above G3-LQ (56.90 / 45.58).
- On Nr3D/Sr3D, LIBA scores 64.5% (Nr3D) and 75.8% (Sr3D) overall, best among methods without 3D-VL pretraining, comparable to fine-tuned GPS(F) and 3D-VisTA(F).
- Ablation on ScanRefer shows the full model (DBA+LCOM+LLM-HQS) improves Acc@0.25 from 54.59 to 59.55 over the baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LIBA instead of MCLN | 3D visual grounding · ScanRefer Overall | accuracy IoU@0.25: 57.17 -> 59.57 | Helped |
| use LIBA instead of G3-LQ | 3D visual grounding · ScanRefer Overall | accuracy IoU@0.5: 45.58 -> 48.96 | Helped |
| use LIBA instead of GPS(F) | 3D visual grounding · Nr3D All | accuracy: 64.7 -> 64.5 | Hurt |
| remove DBA, LCOM, and LLM-HQS (ablation) | LIBA · ScanRefer Overall | accuracy IoU@0.25: 54.59 -> 59.55 | Hurt |
| add DBA only; omit LCOM and LLM-HQS (ablation) | LIBA · ScanRefer Overall | accuracy IoU@0.25: 56.37 -> 59.55 | Hurt |
Surprising findings
- LLM-HQS alone yields only modest gains (55.10 vs 54.59 baseline), with the authors arguing DBA-level multi-granularity alignment is needed for it to help.
Limitations
- LLM-HQS relies on a rule-based scene graph parser for relation reasoning.
- Comparisons with 3D-VL pretraining methods are not like-for-like; the strongest Nr3D/Sr3D numbers rely on fine-tuning or pretraining.
- Base LLM and its size are not specified in the text provided.
Experimental setup
Datasets: ScanRefer Nr3D Sr3D
Benchmarks: ScanRefer Nr3D Sr3D
Metrics: accuracy@IoU0.25 accuracy@IoU0.5 overall accuracy
Baselines: EDA G3-LQ BUTD-DETR 3D-VisTA GPS MCLN 3D-SPS 3DVG
Builds on: DETR-style 3D-VG (EDA, BUTD-DETR, G3-LQ) LISA PointNet++ DeBERTa GroundingDINO
Closest papers by meaning
- G^3-LQ: Marrying Hyperbolic Alignment with Explicit Semantic-Geometric Modeling for 3D Visual Grounding (CVPR 2024 · similarity 0.90)
- Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysis (CVPR 2025 · similarity 0.89)
- MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations (NeurIPS 2024 · similarity 0.89)
- DenseGrounding: Improving Dense Language-Vision Semantics for Ego-centric 3D Visual Grounding (ICLR 2025 · similarity 0.89)
- Language-Image Models with 3D Understanding (ICLR 2025 · similarity 0.89)
- ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.