Research map / Point clouds / Point cloud classification
ProxyTransformation: Preshaping Point Cloud Manifold With Proxy Attention For 3D Visual Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Proxy Transformation, which uses text and image features as proxy tokens in a linear-complexity Proxy Attention to transform point cloud submanifolds for ego-centric 3D visual grounding on EmbodiedScan.
Problem
Ego-centric 3D visual grounding from multi-view RGB-D point clouds suffers from background redundancy and depth noise that distort target-region manifolds, while existing point cloud enhancement methods are too preprocessing-heavy for real-time use.
Approach
Deformable point clustering finds target submanifolds, and a Proxy Attention block (attention compressed through proxy tokens) lets text features predict per-cluster translations and image features predict per-cluster linear transformations, enhancing the point cloud online without offline scene reconstruction.
Key results
- Overall AP25 of 41.08 vs 33.59 for EmbodiedScan (about 7.49% improvement), trained on mini split.
- Overall AP50 of 19.00 vs 14.40 for EmbodiedScan (about 4.60% improvement).
- Proxy Attention reduces attention-block FLOPs by 40.6% (8.36G to 4.97G) while improving overall mAP from 40.14 to 41.08.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace DenseG with ProxyTransformation | 3D visual grounding · EmbodiedScan val Overall | AP25: 39.70 -> 41.08 | Helped |
| replace DenseG with ProxyTransformation | 3D visual grounding · EmbodiedScan val Overall | AP50: 18.31 -> 19.00 | Helped |
| add Proxy Transformation without deformable point clustering (ablation) | baseline with implementation-detail enhancements · EmbodiedScan val Overall | mAP (IoU > 25%): 36.53 -> 39.76 | Helped |
| add deformable offsets (ablation) | Proxy Transformation with Grid Prior · EmbodiedScan val Overall | mAP (IoU > 25%): 39.76 -> 39.91 | Helped |
| add Grid Prior (ablation) | Proxy Transformation with deformable offsets · EmbodiedScan val Hard | mAP (IoU > 25%): 32.18 -> 34.38 | Helped |
| replace vanilla self-attention with Proxy Block (ablation) | 3D visual grounding attention blocks · EmbodiedScan val Overall | mAP (IoU > 25%): 40.14 -> 41.08 | Helped |
| replace vanilla self-attention with Proxy Block (ablation) | 3D visual grounding attention blocks · EmbodiedScan val | FLOPs: 8.36G -> 4.97G | Helped |
| use grid size 10 instead of no grid prior (ablation) | ProxyTransformation · EmbodiedScan val Overall | mAP (IoU > 25%): 39.91 -> 40.29 | Helped |
| use grid size 12 instead of grid size 10 (ablation) | ProxyTransformation · EmbodiedScan val Overall | mAP (IoU > 25%): 40.29 -> 41.08 | Helped |
| increase grid size from 12 to 14 (ablation) | ProxyTransformation · EmbodiedScan val Overall | mAP (IoU > 25%): 41.08 -> 40.56 | Hurt |
| use 1 Proxy layer instead of 3 (ablation) | ProxyTransformation · EmbodiedScan val Overall | mAP (IoU > 25%): 41.08 -> 40.99 | Hurt |
| use 2 Proxy layers instead of 3 (ablation) | ProxyTransformation · EmbodiedScan val Overall | mAP (IoU > 25%): 41.08 -> 40.79 | Hurt |
Surprising findings
- Surpasses baselines trained on the full training set while training only on the approximately 20% mini split.
- Proxy Transformation scores lower than DenseG on the AP50 Hard metric (14.09 vs 15.88).
Limitations
- Trained only on a mini split (about 20% of data) due to limited resources, so full-data scaling is untested.
- Gains on hard AP50 are not consistent with DenseG (14.09 vs 15.88 on Hard AP50).
Experimental setup
Datasets: EmbodiedScan
Benchmarks: EmbodiedScan
Metrics: AP25 AP50 mAP
Baselines: EmbodiedScan DenseG ScanRefer BUTD-DETR L3Det Vanilla self-attention
Builds on: EmbodiedScan Transformer deformable convolution PointNet CLIP
Closest papers by meaning
- PointMapPolicy: Structured Point Cloud Processing for Multi-Modal Imitation Learning (NeurIPS 2025 · similarity 0.86)
- S$^2$-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance (CVPR 2026 · similarity 0.86)
- On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation (NeurIPS 2025 · similarity 0.86)
- DenseGrounding: Improving Dense Language-Vision Semantics for Ego-centric 3D Visual Grounding (ICLR 2025 · similarity 0.86)
- GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model (ICML 2025 · similarity 0.85)
- Sonata: Self-Supervised Learning of Reliable Point Representations (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.