Research map / 3D vision / Gaussian splatting
CoGS: Controllable Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends 3D Gaussian Splatting to monocular dynamic scenes with per-Gaussian deformation MLPs and regularizers, then adds learned mask-based control signals for real-time, controllable manipulation of articulated parts.
Problem
Monocular dynamic scene reconstruction and re-animation of articulated objects is costly with NeRFs, existing dynamic Gaussian methods need synchronized multi-view cameras, and prior controllable NeRFs require pre-computed control signals and masks.
Approach
Learns per-Gaussian offset MLPs for position, color, rotation and scale over time, regularized by neighborhood rigidity/trajectory losses, then learns a 3D Gaussian-level mask and an unsupervised PCA-based control signal extracted from Gaussian motion to drive per-part attribute offsets.
Key results
- On the 360-degree synthetic dynamic dataset, CoGS reaches 37.90 PSNR and 0.983 SSIM, versus 32.67 PSNR for the best NeRF baseline TiNeuVox-B and 23.07 PSNR for static 3D GS.
- On real dynamic scenes from HyperNeRF, CoGS achieves 29.6 PSNR and 0.950 SSIM, comparable to HyperNeRF (29.8 PSNR, 0.954 SSIM) and better than 3D GS (22.1 PSNR).
- The method achieves controllable manipulation of articulated parts (face eyes/mouth, toy car hood/trunk/doors) without pre-computed control signals or masks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Lnorm (ablation) | CoGS dynamic GS · Synthetic dynamic scenes | PSNR: 37.41 -> 37.90 | Helped |
| add Ldiff (ablation) | CoGS dynamic GS · Synthetic dynamic scenes | PSNR: 37.68 -> 37.90 | Helped |
| add Lrigid (ablation) | CoGS dynamic GS · Synthetic dynamic scenes | PSNR: 37.75 -> 37.90 | Helped |
| add Lnorm (ablation) | CoGS dynamic GS · Real dynamic scenes | PSNR: 29.1 -> 29.6 | Helped |
| add Ldiff (ablation) | CoGS dynamic GS · Real dynamic scenes | PSNR: 29.8 -> 29.6 | Hurt |
| add Lrigid (ablation) | CoGS dynamic GS · Real dynamic scenes | PSNR: 29.4 -> 29.6 | Helped |
Surprising findings
- Static 3D GS scores far below NeRF-based dynamic baselines on the real scenes (22.1 vs 29.8 PSNR), showing that dynamic modeling is essential.
Limitations
- Struggles with shiny or intricately lit objects common to Gaussian splatting pipelines.
- Non-rigid deformation and large-scale motion in monocular settings are difficult.
- PCA-based control signal extraction may fail on highly complex movements.
Experimental setup
Datasets: D-NeRF synthetic dynamic dataset HyperNeRF real dynamic scenes CoNeRF real face scene self-captured toy car scenes
Benchmarks: D-NeRF synthetic dynamic scenes HyperNeRF real dynamic scenes
Metrics: PSNR SSIM LPIPS
Baselines: NeRF DirectVoxGo Plenoxels T-NeRF D-NeRF TiNeuVox 3D GS NV NSFF Nerfies HyperNeRF
Builds on: 3D Gaussian Splatting HyperNeRF CoNeRF Dynamic 3D Gaussians
Closest papers by meaning
- MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.90)
- COLMAP-Free 3D Gaussian Splatting (CVPR 2024 · similarity 0.90)
- SfM-Free 3D Gaussian Splatting via Hierarchical Training (CVPR 2025 · similarity 0.89)
- Mani-GS: Gaussian Splatting Manipulation with Triangular Mesh (CVPR 2025 · similarity 0.89)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.89)
- H3D-DGS: Exploring Heterogeneous 3D Motion Representation for Deformable 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.