Research map / Point clouds / Point cloud classification

Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence

NeurIPS 2026 · Point clouds › Point cloud classification · approach: Point-language and open-vocabulary 3D · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Tango3D aligns 2D patch features from a frozen VGGT with 3D VAE tokens in a shared space, using local dense contrastive loss plus global retrieval loss under a three-stage progressive training schedule.

Problem

Existing 3D foundation models align point clouds only globally to frozen CLIP-like spaces, so they cannot establish fine-grained object-level pixel-to-point correspondence while also supporting global retrieval.

Approach

Use a geometry-aware 2D backbone (VGGT) as the alignment anchor and a pretrained 3D VAE to produce 3D tokens, map both into one shared token space trained with a bidirectional local InfoNCE loss plus a global cross-modal contrastive loss, and stage the training to stabilize joint optimization.

Key results

• Pixel-to-point LocAcc@1 reaches 77.37 with four random views (Stage III), above 75 across all settings, versus VGGT at 72.04 with orthographic views.
• Image-to-shape retrieval with four random views achieves R@1 39.12 and MRR 50.69, exceeding the baselines in single-view R@1 (OpenShape 36.44) only partly; single-view MRR 47.17 trails OpenShape's 48.19.
• Progressive ablation shows LocAcc@1 rising from 76.47 (Stage I) to 77.37 (Stage III) and MRR from 47.91 (Stage II) to 50.69 (Stage III).

Surprising findings

• The originally non-semantic 3D encoder acquires discriminative semantic structure through cross-modal training alone.
• Single-view retrieval still trails OpenShape on several metrics, attributed to DINO class tokens having weaker category discriminability than CLIP.

Limitations

• Local alignment is limited by VGGT patch size (14x14 pixels) and the 1024 3D VAE tokens, adding sub-patch localization error.
• Global branch relies on DINO class tokens, which are less category-discriminative than CLIP features.
• Single-view retrieval still trails OpenShape on several metrics.

Experimental setup

Datasets: Objaverse Objaverse-LVIS ModelNet40

Benchmarks: custom ModelNet40 textured benchmark (pixel-to-point) custom ModelNet40 textured benchmark (image-to-shape retrieval)

Metrics: LocAcc@k Recall@k MRR

Baselines: VGGT VGGT-OBJ Uni3D-g OpenShape ULIP-2

Builds on: VGGT CLIP InfoNCE SAM DINOv2

Closest papers by meaning

• Escaping Plato's Cave: Towards the Alignment of 3D and Text Latent Spaces (CVPR 2025 · similarity 0.88)
• Uni3D: Exploring Unified 3D Representation at Scale (ICLR 2024 · similarity 0.88)
• G^3-LQ: Marrying Hyperbolic Alignment with Explicit Semantic-Geometric Modeling for 3D Visual Grounding (CVPR 2024 · similarity 0.87)
• ImageNet3D: Towards General-Purpose Object-Level 3D Understanding (NeurIPS 2024 · similarity 0.87)
• SemAlign3D: Semantic Correspondence between RGB-Images through Aligning 3D Object-Class Representations (CVPR 2025 · similarity 0.87)
• Proxy3D: Efficient 3D Representations for Vision-Language Models via Semantic Clustering and Alignment (CVPR 2026 · similarity 0.87)