Research map / 3D vision / Depth and stereo

DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers

NeurIPS 2026 · 3D vision › Depth and stereo · approach: Cost-volume multi-view stereo · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Identifies that PRoPE's value-output camera encoding couples rotation and translation in the same dimensions, causing late-stage stagnation in scaled-up NVS training, and proposes DPPE, which decouples them (additive tAdd or dual-projection variants) for stable scaling and better extrapolation.

Problem

Camera-based positional encodings (PRoPE/GTA) stagnate and degrade in late stages of scaled-up training for novel view synthesis, and extrapolation to more views or zoom-in is weak.

Approach

Decouple rotation and translation in the value-output camera positional encoding: tAdd assigns rotation (K R) and translation (additive t) to disjoint feature dimensions with separate attention blocks; dual uses the inverse-transpose projection matrix, which keeps rotation in the first three dims and isolates translation in the fourth.

Key results

• On MVImgNet2 (2 views, 320k iters, mean of 3 seeds), DPPE-tAdd reaches 24.02 PSNR / 0.730 SSIM / 0.180 LPIPS in value-output-only setting vs. 22.91 PSNR for PRoPE.
• PRoPE PSNR peaks around 184k iterations and degrades to 23.14 at 320k, while DPPE-dual@320k reaches 23.92 PSNR.
• Gains are marginal on SpatialVidHQ (19.28 vs 19.21 PSNR), where camera trajectories are simple; the paper acknowledges no clear advantage in such regimes.

Surprising findings

• Applying tAdd to query-key pairs degrades performance relative to applying it only to value-output pairs, suggesting the scene-scale-dependent translation term harms attention similarity.
• Under PRoPE, even best-checkpoint early stopping (184k-240k) falls short of DPPE at the final 320k iteration.

Limitations

• Gains are marginal or absent on datasets with simple camera trajectories such as SpatialVidHQ.
• The identifiability hypothesis is supported by per-token analysis but the link to training dynamics is hypothesized rather than fully proven.
• Results are on a relatively small 12-layer, 768-dim model with 2-4 input views and a single target view.

Experimental setup

Datasets: MVImgNet2 RealEstate10K SpatialVidHQ CO3D WildRGBD MegaSynth ARKitScenes

Benchmarks: MVImgNet2 RealEstate10K SpatialVidHQ CO3D WildRGBD MegaSynth

Metrics: PSNR SSIM LPIPS

Baselines: PRoPE GTA Plücker raymaps CaPE UCPE

Builds on: PRoPE GTA RoPE UCPE VGGT

Closest papers by meaning

• Cameras as Relative Positional Encoding (NeurIPS 2025 · similarity 0.90)
• GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformers (ICLR 2024 · similarity 0.88)
• Less Decoder is More Encoder: Geometric Representation Learning from Novel View Synthesis (NeurIPS 2026 · similarity 0.87)
• Parabolic Position Encoding: Vision-Centric, Principled, Extrapolatable, General (NeurIPS 2026 · similarity 0.86)
• Understanding Multi-View Transformers (NeurIPS 2026 · similarity 0.86)
• LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate (NeurIPS 2024 · similarity 0.86)