Research map / 3D vision / NeRF and radiance fields

SCION: Scene Composition via Instanced Neural Primitives

NeurIPS 2026 · 3D vision › NeRF and radiance fields · approach: Inverse rendering with neural fields

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SCION, a hierarchical Gaussian scene representation with a shared vocabulary of primitives and lightweight world-space instances, fit end-to-end from multi-view images for compact, editable, animatable reconstructions.

Problem

3D Gaussian Splatting fits millions of independent Gaussians per scene, which is large and structureless, making edits and animation hard and ignoring repeated elements in real scenes.

Approach

Replace independent Gaussians with a compact library of reusable local-frame primitives (bundles of Gaussians) plus lightweight world-space instances that select and place transformed primitives, jointly optimized with primitive-local densification, instance densification (clone/subdivide), and an adversarial patch loss preserving sharpness under sharing.

Key results

• At matched ~1.2-1.5 MB on MipNeRF360, Ours-Small reaches 22.01 PSNR / 0.605 SSIM / 0.431 LPIPS at 1.204 MB, versus HAC++ at 26.51 PSNR and 1.699 MB.
• Ours-Small uses 1.20, 1.55, 0.73, and 0.16 MB on MipNeRF360, Tanks & Temples, Deep Blending, and Synthetic-NeRF, with 22.00, 18.61, 24.02, and 24.39 dB PSNR respectively.
• Adversarial training yields the largest qualitative gain; on the garden scene the full Ours-Large variant reaches 0.27 LPIPS versus 0.33 before adding adversarial loss, at a small PSNR cost.

Surprising findings

• Adding the LPIPS loss, the metric being evaluated, did not yield as large an LPIPS gain as adversarial training, suggesting reference-based losses alone reach local minima.
• The learning objective trades PSNR and SSIM for perceptual quality, lowering image metrics relative to unstructured compression baselines.

Limitations

• Benefits depend on real repetition; scenes with unique geometry or strong per-instance variation underfit or waste capacity on one-off templates.
• Hard primitive assignment is fixed during reconstruction, preventing instances from moving between primitives.
• Training is more expensive than vanilla 3DGS due to two-pass optimization, discriminator, LPIPS loss, and extra density control.

Experimental setup

Datasets: Mip-NeRF 360 Tanks & Temples Deep Blending Synthetic-NeRF

Benchmarks: Mip-NeRF 360 Tanks & Temples Deep Blending Synthetic-NeRF

Metrics: PSNR SSIM LPIPS model size (MB)

Baselines: ContextGS HAC++ OMG PCGS 3DGS

Builds on: 3D Gaussian Splatting MH-3DGS HAC++ Splat and Replace

Closest papers by meaning

• The Scene Language: Representing Scenes with Programs, Words, and Embeddings (CVPR 2025 · similarity 0.86)
• GaussianBlock: Building Part-Aware Compositional and Editable 3D Scene by Primitives and Gaussians (ICLR 2025 · similarity 0.86)
• Compositional Scene Understanding through Inverse Generative Modeling (ICML 2025 · similarity 0.86)
• DORSal: Diffusion for Object-centric Representations of Scenes $\textit{et al.}$ (ICLR 2024 · similarity 0.85)
• Decompositional Neural Scene Reconstruction with Generative Diffusion Prior (CVPR 2025 · similarity 0.85)
• DyST: Towards Dynamic Neural Scene Representations on Real-World Videos (ICLR 2024 · similarity 0.85)