Research map / 3D vision / Gaussian splatting
Morpheus: Text-Driven 3D Gaussian Splat Shape and Color Stylization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Morpheus, an autoregressive text-driven stylization pipeline for 3D Gaussian Splatting scenes that changes both color and geometry using an RGBD diffusion model, a Warp ControlNet, and depth-guided feature sharing.
Problem
Novel-view stylization of 3D Gaussian Splat scenes from text prompts usually only changes texture and cannot convincingly alter geometry while keeping multi-view consistency.
Approach
An RGBD latent diffusion model with separate color and depth denoising schedules (giving independent appearance and shape strength control), plus a Warp ControlNet conditioned on warped stylized composites and depth-guided cross-attention and feature injection to propagate stylization across frames autoregressively.
Key results
- Ours achieves CLIP Direction Similarity 0.175 vs 0.123 for GaussCtrl and 0.098 for Instruct NeRF2NeRF across 53 stylizations.
- Ours reports CLIP Direction Consistency 0.606, RMSE 0.0370 and LPIPS 0.0378, best RMSE among compared methods.
- In an A/B user study with 31 participants, users preferred the method for style adherence and aesthetic quality over baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Morpheus instead of the strongest baseline, DGE | novel-view stylization · 53 stylizations | CLIP Direction Similarity: .113 -> .175 | Helped |
| Use Morpheus instead of the strongest baseline, GaussCtrl | novel-view stylization · 53 stylizations | CLIP Direction Consistency: .590 -> .606 | Helped |
| Use Morpheus instead of the strongest baseline, DGE | novel-view stylization · 53 stylizations | RMSE: .0384 -> .0370 | Helped |
| Use Morpheus instead of the strongest baseline, Instruct-GS2GS | novel-view stylization · 53 stylizations | LPIPS: .0403 -> .0378 | Helped |
| remove consistency mechanisms and stylize frames independently (ablation) | Morpheus · Sequential stylized frames | Similarity: 0.175 -> 0.161 | Hurt |
| replace the RGBD model with RGB inpainting and depth ControlNet (ablation) | Morpheus · Sequential stylized frames | Similarity: 0.175 -> 0.110 | Hurt |
| add DAv2 depth prediction to the warp and inpainting pipeline (ablation) | Morpheus · Sequential stylized frames | Similarity: 0.175 -> 0.104 | Hurt |
| remove depth-informed feature sharing (ablation) | Morpheus · Sequential stylized frames | Similarity: 0.175 -> 0.178 | Helped |
| use full-frame cross-attention without feature injection (ablation) | Morpheus · Sequential stylized frames | Similarity: 0.175 -> 0.180 | Helped |
| remove Warp ControlNet and use inpainting (ablation) | Morpheus · Sequential stylized frames | Similarity: 0.175 -> 0.170 | Hurt |
Surprising findings
- Stylizations with higher depth strength change geometry while keeping overall color gamut, and the paper argues CLIP metrics alone are unreliable for stylization quality.
Limitations
- Stylization quality depends on trajectory selection.
- Inherits errors from the original 3DGS model, such as fuzzy regions and incorrect depth.
- Relies on the ~10 minute runtime, which could be improved with faster diffusion models.
Experimental setup
Datasets: ReLAION-2B aesthetic subset Instruct-NeRF2NeRF scenes GaussCtrl scenes ScanNet++ Mip-NeRF360
Benchmarks: Instruct-NeRF2NeRF scenes GaussCtrl scenes ScanNet++ Mip-NeRF360
Metrics: CLIP Direction Similarity CLIP Direction Consistency RMSE LPIPS Sequential RMSE Sequential LPIPS user preference
Baselines: Instruct NeRF2NeRF Instruct GS2GS GaussCtrl DGE ConsistDreamer
Builds on: Stable Diffusion 2.1 ControlNet 3D Gaussian Splatting GaussCtrl Instruct-NeRF2NeRF
Closest papers by meaning
- Multi-StyleGS: Stylized Gaussian Splatting with Multiple Styles (AAAI 2025 · similarity 0.92)
- ReGS: Reference-based Controllable Scene Stylization with Gaussian Splatting (NeurIPS 2024 · similarity 0.92)
- CLIPGaussian: Universal and Multimodal Style Transfer Based on Gaussian Splatting (NeurIPS 2025 · similarity 0.90)
- Styl3R: Instant 3D Stylized Reconstruction for Arbitrary Scenes and Styles (NeurIPS 2025 · similarity 0.90)
- SGSST: Scaling Gaussian Splatting Style Transfer (CVPR 2025 · similarity 0.89)
- Text-to-3D using Gaussian Splatting (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.