Research map / 3D vision / Depth and stereo

DarkVGGT: Seeing Through Darkness Using Thermal Geometry without Daylight Tax

NeurIPS 2026 · 3D vision › Depth and stereo · approach: Feed-forward 3D reconstruction · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DarkVGGT, a VGGT-based RGB-thermal feed-forward geometry model that uses physics-inspired thermal factorization and geometry-shared thermal routing to improve dark-scene depth and pose while preserving well-lit RGB performance.

Problem

Feed-forward 3D geometry models trained on well-lit RGB degrade in dark, low-visibility scenes, and naive RGB-thermal adaptation can erode the pretrained daylight geometry prior ("daylight tax").

Approach

Factorizes thermal tokens into emissive (pseudo-emissivity gated) and reflective high-pass components, and routes only geometry-shared thermal content into the RGB stream via stop-gradient distillation and reliability gating, with thermal-dropout RGB-prior anchoring to a frozen VGGT.

Key results

• On low-light RGB-T benchmarks, DarkVGGT reduces average AbsRel from 0.2010 (SEAR) to 0.1373 and raises delta<1.25 from 0.6901 to 0.8243.
• Average pose AUC@30 rises from 47.7 (SEAR) to 66.3 for DarkVGGT across ViViD++, STheReO, and Dark3R.
• On well-lit ETH3D and ScanNet++ (RGB-only), DarkVGGT stays close to VGGT, e.g., ScanNet++ delta<1.25 of 0.9608 vs 0.9479 for VGGT, and ETH3D 0.9584 vs 0.9729.

Surprising findings

• Adding plain LoRA to the AA blocks yields only marginal gains over full AA fine-tuning, while the physics-inspired thermal factorization gives the largest standalone improvement.
• Removing the sparsity regularizer improves AUC@30 (66.3 to 69.2) while degrading depth accuracy.

Limitations

• Assumes moderately aligned paired RGB-T inputs; weakly aligned or unpaired settings not handled.
• Several hyperparameters for residual scaling, sparsity, and fusion budgeting need validation-based or sensor-specific tuning.
• Dark3R evaluation uses pseudo-thermal inputs synthesized by ThermalGen rather than measured thermal images.

Experimental setup

Datasets: ViViD++ STheReO Dark3R ETH3D ScanNet++

Benchmarks: ViViD++ STheReO Dark3R ETH3D ScanNet++

Metrics: AbsRel delta<1.25 RMSE_log RRA@30 RTA@30 AUC@30

Baselines: DUSt3R MASt3R Dark3R DepthAnything3 VGGT SEAR

Builds on: VGGT DUSt3R SEAR LoRA DINOv2

Closest papers by meaning

• Geometry Meets Light: Leveraging Geometric Priors for Universal Photometric Stereo Under Limited Multi-Illumination Cues (AAAI 2026 · similarity 0.87)
• Thermal is Always Wild: Characterizing and Addressing Challenges in Thermal-Only Novel View Synthesis (CVPR 2026 · similarity 0.87)
• ThermalGaussian: Thermal 3D Gaussian Splatting (ICLR 2025 · similarity 0.86)
• EAG3R: Event-Augmented 3D Geometry Estimation for Dynamic and Extreme-Lighting Scenes (NeurIPS 2025 · similarity 0.86)
• Sparse Views Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo (CVPR 2024 · similarity 0.86)
• Lighting Every Darkness with 3DGS: Fast Training and Real-Time Rendering for HDR View Synthesis (NeurIPS 2024 · similarity 0.85)