Research map / 3D vision / Depth and stereo

Less Decoder is More Encoder: Geometric Representation Learning from Novel View Synthesis

NeurIPS 2026 · 3D vision › Depth and stereo · approach: Foundation models for monocular geometry · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SNAP, a self-supervised pose-conditioned multi-view encoder-decoder that predicts frozen DINOv3 latent features of novel views with a receptive-field-restricted (BlockNN) patch decoder, yielding transferable geometric representations without 3D supervision.

Problem

Encoder-based novel view synthesis produces poor geometric representations for downstream tasks because expressive decoders absorb representational load and pixel-space targets emphasize low-level statistics.

Approach

Restrict the decoder's target-view spatial receptive field with a BlockNN mask (RF=1 reduces to a pointwise MLP) and supervise with latent DINOv3 patch features (feature plus gradient L1 loss) instead of RGB pixels, so geometric structure resides in the multi-view encoder.

Key results

• On RealEstate10K point correspondence, SNAP reaches PCK@5px 15.48 vs 12.85 for VGGT and 12.09 for Muskie.
• Relative depth on 7Scenes: SNAP AbsRel 0.9257 vs VGGT 1.9138 and LagerNVS 2.3858 (VGGT is worse here than reported in its own row; SNAP beats NVS encoders).
• Robot manipulation under camera shifts: SNAP outperforms its frozen DINOv3 teacher and tracks VGGT, while DINOv3 and ResNet collapse beyond 0.2 radians.

Surprising findings

• Larger decoder receptive field lowers reconstruction error but degrades downstream correspondence, localization and pose performance.
• Predicting raw RGB or early-layer DINOv3 targets degrades geometric transfer even though it gives easier reconstruction.

Limitations

• Pretraining limited to static, textured scenes and posed video; no dynamic scenes.
• Results bounded by fixed compute and data budget; scaling behavior not characterized.
• Still trails VGGT on some metrics (e.g., DL3DV relative pose and some depth configurations).

Experimental setup

Datasets: RealEstate10K DL3DV Co3Dv2 Hypersim 7Scenes MimicGen

Benchmarks: RealEstate10K DL3DV Hypersim 7Scenes MimicGen

Metrics: PCK APE AUC rotation accuracy translation accuracy AbsRel delta1 success rate

Baselines: LVSM RayZer ERayZer LagerNVS Muskie VGGT Adapt3R DINOv3

Builds on: PRoPE DINOv3 LVSM Muskie RoPE2D

Closest papers by meaning

• Learning To Render Novel Views From Wide-Baseline Stereo Pairs (CVPR 2023 · similarity 0.89)
• LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias (ICLR 2025 · similarity 0.89)
• GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformers (ICLR 2024 · similarity 0.88)
• Novel View Synthesis with Pixel-Space Diffusion Models (CVPR 2025 · similarity 0.88)
• DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers (NeurIPS 2026 · similarity 0.87)
• NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesis (NeurIPS 2026 · similarity 0.87)