Research map / 3D vision / Face and shape
ShapeShifter: 3D Variations Using Multiscale and Sparse Point-Voxel Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a multiscale diffusion model over sparse voxel grids storing oriented, colored point samples to generate geometry-detailed shape variations from a single 3D exemplar in minutes.
Problem
Generating high-quality, geometrically detailed 3D shape variations from a single reference model, while keeping training and inference fast and avoiding SDF/occupancy-based smoothing.
Approach
Replace SDF/triplane features with explicit per-voxel point, normal, color and mask features stored in a sparse voxel grid (fVDB), with a multiscale diffusion pipeline (SinDDM-style) where each level is trained in parallel and uses a learned level-specific upsampler.
Key results
- ShapeShifter achieves average SSFID G-Qual of 0.21 versus 0.86 for Sin3DM and 8.97 for Sin3DGen across eight exemplars.
- Training takes about 12 minutes plus 6 seconds encoding on an RTX 3080, versus 2.5 hours for Sin3DM.
- Inference totals 10.7 seconds, versus 15.8 seconds for Sin3DM and 139 seconds for Sin3DGen (on A100).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ShapeShifter instead of Sin3DM | single-exemplar 3D shape generation · 8 exemplars (average) | G-Qual. (SSFID): 0.86 -> 0.21 | Helped |
| use ShapeShifter instead of Sin3DGen | single-exemplar 3D shape generation · 8 exemplars (average) | G-Qual. (SSFID): 8.97 -> 0.21 | Helped |
| use ShapeShifter instead of Sin3DM | single-exemplar 3D shape generation · 8 exemplars (average) | G-Div. (pairwise 1-IoU): 0.21 -> 0.21 | No clear effect |
| use ShapeShifter instead of Sin3DM | single-exemplar 3D shape generation · single exemplar | training time: 2.5 hrs -> 12 min | Helped |
| use ShapeShifter instead of Sin3DGen | single-exemplar 3D shape generation · single exemplar | training time: 15 min -> 12 min | Helped |
| use ShapeShifter instead of Sin3DM | single-exemplar 3D shape generation · single variation | inference time: 15.8 sec -> 10.7 sec | Helped |
| use ShapeShifter instead of Sin3DGen | single-exemplar 3D shape generation · single variation | inference time: 139 sec -> 10.7 sec | Helped |
| use ShapeShifter instead of SSG | single-exemplar 3D shape generation · single variation | inference time: 0.1 sec -> 10.7 sec | Hurt |
| add a learned upsampler instead of trilinear interpolation (ablation) | ShapeShifter · shape variation | geometric detail and structural coherence | Helped |
| use point and normal features instead of SDF features (ablation) | ShapeShifter · shape variation at 1283 resolution | geometric detail | Helped |
Surprising findings
- Pairwise IoU diversity rewards artifacts, so diversity scores must be interpreted with care, per the authors.
Limitations
- Variations are restricted to be similar to the exemplar, as with prior patch-based exemplar methods.
- Does not generate fine textures; relies on external super-resolution for texturing.
- Quality-diversity metrics (SSFID, IoU) have blind spots.
Experimental setup
Datasets: 3D textured exemplars from Sin3DM one open-surface exemplar created by the authors
Benchmarks: Sin3DM exemplar set
Metrics: SSFID pairwise IoU diversity training time inference time
Baselines: Sin3DM Sin3DGen SSG
Builds on: SinDDM fVDB Sin3DM SinGAN
Closest papers by meaning
- Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape (ICLR 2024 · similarity 0.88)
- Make-A-Shape: a Ten-Million-scale 3D Shape Model (ICML 2024 · similarity 0.88)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.88)
- MeshFormer : High-Quality Mesh Generation with 3D-Guided Reconstruction Model (NeurIPS 2024 · similarity 0.87)
- Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation (CVPR 2024 · similarity 0.87)
- GPLD3D: Latent Diffusion of 3D Shape Generative Models by Enforcing Geometric and Physical Priors (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.