Research map / 3D vision / Depth and stereo

RegimeVGGT: Layer-Wise Spatially Preserving Redundancy Removal for Visual Geometry Grounded Transformer

NeurIPS 2026 · 3D vision › Depth and stereo · approach: Feed-forward 3D reconstruction · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Training-free, layer-wise U-shaped dual-axis compression (saliency-guided token merging plus selectively protected phase-shifted K/V downsampling) that accelerates VGGT's global attention for dense reconstruction and pose.

Problem

VGGT's quadratic cross-frame global attention limits scalability to long multi-view sequences, and existing training-free accelerators compress uniformly across layers, ignoring layer heterogeneity.

Approach

Diagnoses three depth bands in VGGT (shallow, middle, deep) via probing, causal intervention, and rank spectra, then applies band-specific merge ratios and K/V densities, protecting DINOv2 CLS-saliency tokens, camera/register tokens, and a frame-0 anchor, with a phase-shifted per-frame K/V sub-grid.</idea>

Key results

• On ScanNet-50 at 1000 frames, RegimeVGGT runs in 71.5s versus 477.3s for VGGT* and 113.5s for FastVGGT with CD 0.465 vs 0.463 for VGGT*.
• Abstract and conclusion claim a 6.7x speedup over VGGT* on ScanNet-1000 at matched reconstruction quality.
• On Tanks & Temples, mean AUC@30 is 90.9 (matching FastVGGT) with runtime 111s versus 552.8s for VGGT*.

Surprising findings

• Skipping deep-band global attention preserves dense reconstruction but collapses long-sequence camera pose, so deep cross-frame attention must be kept for camera/register tokens.
• Retaining shallow G2F (geometry-to-frame) attention yields weaker dense-geometry probe signals than removing it.

Limitations

• Per-band compression rates are fixed from scene-pooled diagnostics rather than adapted per input.
• Protected token set relies on frozen DINOv2 CLS saliency; learned geometry-aware selectors are not explored.
• AVGGT is excluded from experiments because its code is not public.

Experimental setup

Datasets: 7 Scenes NRGBD ScanNet Tanks & Temples DTU

Benchmarks: 7 Scenes NRGBD ScanNet-50 Tanks & Temples DTU

Metrics: Accuracy Completeness Normal Consistency Chamfer Distance AUC@30 ATE ARE RPE-rot RPE-trans wall-clock time

Baselines: VGGT* FastVGGT S-VGGT Fast3R CUT3R

Builds on: VGGT ToMe FastVGGT AVGGT

Closest papers by meaning

• VGGT: Visual Geometry Grounded Transformer (CVPR 2025 · similarity 0.91)
• QVGGT: Post-Training Quantized Visual Geometry Grounded Transformer (CVPR 2026 · similarity 0.90)
• FastVGGT: Fast Visual Geometry Transformer (ICLR 2026 · similarity 0.90)
• TurboVGGT: Fast Visual Geometry Reconstruction with Adaptive Alternating Attention (NeurIPS 2026 · similarity 0.89)
• LiteVGGT: Boosting Vanilla VGGT via Geometry-aware Cached Token Merging (CVPR 2026 · similarity 0.89)
• VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold (NeurIPS 2025 · similarity 0.86)