Research map / 3D vision / 3D generation
Styl3R: Instant 3D Stylized Reconstruction for Arbitrary Scenes and Styles
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Styl3R is a feed-forward network that jointly reconstructs and stylizes 3D Gaussian scenes from sparse unposed content images and a single arbitrary style image in under a second, using a dual-branch (structure/appearance) design and an identity loss during stylization fine-tuning.
Problem
Instant 3D stylization of scenes from sparse, unposed content images and an arbitrary style image while preserving multi-view consistency and scene structure, without test-time optimization or dense posed inputs.
Approach
A dual-branch feed-forward model in which a DUSt3R/MASt3R-style structure branch predicts Gaussian geometry, while a separate appearance branch (multi-view global self-attention followed by cross-attention to style tokens) predicts stylized colors; NVS pre-training followed by stylization fine-tuning with the structure branch frozen and an identity loss that keeps photorealistic reconstruction.
Key results
- User study on RE10K: Styl3R received 53.29% of votes versus 24.10% for StyTr2 and 14.13% for ARF.
- ArtFID on RE10K is 35.12 for Styl3R, compared with 38.93 for StyTr2 and 42.95 for ARF.
- NVS on RE10K: Ours-stylization reaches 24.055 PSNR and 0.820 SSIM, versus 25.033 PSNR for NoPoSplat.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use both relu3_1 and relu4_1 instead of relu3_1 alone in content loss (ablation) | Styl3R · Content loss layer ablation | ArtFID: 42.01 -> 35.83 | Helped |
| Use both relu3_1 and relu4_1 instead of relu3_1 alone in content loss (ablation) | Styl3R · Content loss layer ablation | Histogram: 0.277 -> 0.239 | Helped |
| Use both relu3_1 and relu4_1 instead of relu3_1 alone in content loss (ablation) | Styl3R · Content loss layer ablation | votes (%): 35.20 -> 64.80 | Helped |
| Remove identity loss during stylization fine-tuning (ablation) | Styl3R · Novel view synthesis | scene appearance recovery | Hurt |
| Train a model with 2 views and provide 8 input views instead of using the 4-view model (ablation) | Styl3R · 8 input views | multi-view Gaussian alignment | Hurt |
| Stylization fine-tuning | Styl3R · RE10K NVS | PSNR: 24.871 -> 24.055 | Hurt |
| Stylization fine-tuning | Styl3R · RE10K NVS | LPIPS: 0.165 -> 0.179 | Hurt |
| Use Styl3R instead of StyTr2 for 3D stylization | 3D stylization · RE10K | ArtFID: 38.93 -> 35.12 | Helped |
Surprising findings
- Without the identity loss during stylization fine-tuning, the model fails to recover the original scene color tone during NVS.
- A 2-view model fails to align Gaussians with 8 input views, while the 4-view-trained model generalizes to 8 views without 8-view training.
Limitations
- Limited to 2-8 input views and 256x256 resolution, inherited from the MASt3R-based base model.
- Supports only static scenes; dynamic scenes are future work.
- Feed-forward reconstruction quality is lower than optimization-based two-stage pipelines, and scalability is limited by VRAM.
Experimental setup
Datasets: RealEstate10K DL3DV WikiArt Tanks and Temples
Benchmarks: RealEstate10K Tanks and Temples
Metrics: multi-view consistency (LPIPS/RMSE warped) ArtFID RGB-uv histogram user study votes PSNR SSIM LPIPS
Baselines: AdaIN AdaAttN StyTr2 ARF StyleRF StyleGaussian NoPoSplat pixelSplat
Builds on: DUSt3R MASt3R NoPoSplat AdaIN
Closest papers by meaning
- PNeSM: Arbitrary 3D Scene Stylization via Prompt-Based Neural Style Mapping (AAAI 2024 · similarity 0.91)
- ReGS: Reference-based Controllable Scene Stylization with Gaussian Splatting (NeurIPS 2024 · similarity 0.90)
- Morpheus: Text-Driven 3D Gaussian Splat Shape and Color Stylization (CVPR 2025 · similarity 0.90)
- Multi-StyleGS: Stylized Gaussian Splatting with Multiple Styles (AAAI 2025 · similarity 0.89)
- FPRF: Feed-Forward Photorealistic Style Transfer of Large-Scale 3D Neural Radiance Fields (AAAI 2024 · similarity 0.89)
- 3DToonify: Creating Your High-Fidelity 3D Stylized Avatar Easily from 2D Portrait Images (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.