Research map / 3D vision / NeRF and radiance fields
ShowMak3r: Compositional TV Show Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents ShowMak3r, a pipeline that reconstructs editable dynamic 3D Gaussian scenes (stage plus multiple actors) from shot-changing TV show clips, using depth-guided stage reconstruction, SMPL actor placement, cross-shot actor matching, and a face-fitting residual network.
Problem
Reconstructing dynamic radiance fields from monocular TV show clips is hard due to occluding actors, diverse facial expressions, cluttered stages, narrow baselines, and abrupt shot changes.
Approach
Combines depth-aligned 3DGS stage reconstruction with a 3DLocator that places SMPL actors on the stage by aligning posed vertices to monocular depth, a ShotMatcher that associates actors across shot boundaries by 3D distance, and an MLP residual on colors and opacity for facial detail, yielding a compositional, editable scene.
Key results
- Stage reconstruction on TBBT living room reaches 19.65 PSNR vs 19.21 for vanilla 3DGS and 18.81 for Sitcoms3D.
- Face-fitting refinement improves face crop PSNR to 24.34 vs 20.17 for ExAvatar.
- 3DLocator reduces MTED from 1.99 to 1.19 and MPED from 0.24 to 0.10 across shot boundaries.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add face-fitting refinement (ablation) | ShowMak3r actor reconstruction · Face reconstruction | PSNR: 21.52 -> 24.34 | Helped |
| add face-fitting refinement (ablation) | ShowMak3r actor reconstruction · Face reconstruction | SSIM: 0.82 -> 0.84 | Helped |
| add face-fitting refinement (ablation) | ShowMak3r actor reconstruction · Face reconstruction | LPIPS: 0.36 -> 0.28 | Helped |
| add 3DLocator (ablation) | ShowMak3r actor placement · Two consecutive shots | MTED: 1.99 -> 1.19 | Helped |
| add 3DLocator (ablation) | ShowMak3r actor placement · Two consecutive shots | MPED: 0.24 -> 0.10 | Helped |
Limitations
- Relies on an off-the-shelf 4D human pose estimator that becomes noisy under severe occlusion.
- Monocular depth estimation struggles with zoomed-in close-up shots.
Experimental setup
Datasets: Sitcoms3D
Benchmarks: Sitcoms3D
Metrics: PSNR SSIM LPIPS MTED MPED
Baselines: HUGS Shape-of-Motion MonST3R Sitcoms3D 3DGS GS-W FSGS ExAvatar
Builds on: 3D Gaussian Splatting Sitcoms3D HUGS Shape-of-Motion Depth Anything
Closest papers by meaning
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.84)
- Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpses (CVPR 2024 · similarity 0.84)
- PAI-Actor: Cinematic Multi-Actor Character Replacement in Dynamic Scenes (NeurIPS 2026 · similarity 0.84)
- Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos (NeurIPS 2025 · similarity 0.83)
- Continuous 3D Perception Model with Persistent State (CVPR 2025 · similarity 0.83)
- ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.