Research map / 3D vision / Face and shape
Ghost on the Shell: An Expressive Representation of General 3D Shapes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Ghost-on-the-Shell (G-Shell), a grid-based differentiable representation that jointly parameterizes a watertight SDF template and an mSDF defining open surfaces, enabling rasterization-based multiview reconstruction and diffusion-based generation of non-watertight meshes.
Problem
Modeling general 3D shapes including thin, open, non-watertight surfaces is hard: UDF-based methods need error-prone isosurface extraction and implicit rendering is slow, while mesh generators handle only watertight shapes.
Approach
Parameterize open surfaces as islands on a learnable watertight SDF template via a manifold signed distance field (mSDF) stored on the same grid; extract the SDF isosurface and mSDF isoline with a Marching-Cubes-like lookup table, enabling differentiable rasterization and diffusion over grid attributes.
Key results
- On DeepFashion3D reconstruction, Nvdiffrecmc w/ G-Shell (FC) reaches avg PSNR 33.277 vs 31.074 for NeUDF and average Chamfer distance 0.177 cm vs 0.286 for NeUDF (G-Shell tetrahedral).
- Reconstruction time is about 3 hours per shape versus 17.3 h for NeuralUDF and 16.4 h for NeUDF; inference runs at 2.7 s/img versus 1.8 min/img for NeuralUDF.
- G-MeshDiffusion achieves MV-FID of 79.01 on lower garments and 88.98 on upper garments, versus 191.09 and 167.69 for MeshDiffusion, and generally beats MeshDiffusion and GET3D on MMD/COV/1-NNA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace DMTet with G-Shell for reconstruction | Nvdiffrecmc · DeepFashion3D garments | PSNR: 31.677 -> 33.039 | Helped |
| replace DMTet with G-Shell for reconstruction | Nvdiffrecmc · DeepFashion3D garments | Chamfer distance (cm): 0.662 -> 0.177 | Helped |
| use FlexiCubes instead of tetrahedral grids (ablation) | G-Shell reconstruction · DeepFashion3D garments | PSNR: 33.039 -> 33.277 | Helped |
| use FlexiCubes instead of tetrahedral grids (ablation) | G-Shell reconstruction · DeepFashion3D garments | Chamfer distance (cm): 0.177 -> 0.203 | Hurt |
| use G-MeshDiffusion rather than MeshDiffusion | mesh generation · Lower Garments | MV-FID: 191.09 -> 79.01 | Helped |
| use G-MeshDiffusion rather than GET3D | mesh generation · Lower Garments | MV-FID: 95.69 -> 79.01 | Helped |
| use G-MeshDiffusion rather than MeshDiffusion | mesh generation · Upper Garments | MV-FID: 167.69 -> 88.98 | Helped |
| use G-MeshDiffusion rather than GET3D | mesh generation · Upper Garments | MV-FID: 112.45 -> 88.98 | Helped |
| increase metalness from 0 to 1 in the target material (ablation) | G-Shell reconstruction · Metalness 0–1, sweater | PSNR: 36.01 -> 32.31 | Mixed |
Surprising findings
- Watertight-only baseline Nvdiffrecmc with DMTet outperforms the implicit UDF/SDF non-watertight baselines on several garment instances in PSNR.
Limitations
- Cannot model self-intersecting or non-orientable surfaces such as Mobius strips.
- Requires higher grid resolution to model tiny topological holes.
- 3D U-Nets used for diffusion are memory-inefficient and hard to scale to high resolution.
Experimental setup
Datasets: DeepFashion3D Cloth3D NeRF synthetic
Benchmarks: DeepFashion3D Cloth3D
Metrics: PSNR Chamfer distance MMD COV 1-NNA MV-FID
Baselines: NeuralUDF NeUDF NeAT NeuS Nvdiffrecmc w/ DMTet MeshDiffusion GET3D
Builds on: DMTet Nvdiffrecmc MeshDiffusion FlexiCubes MeshUDF
Closest papers by meaning
- Physically Compatible 3D Object Modeling from a Single Image (NeurIPS 2024 · similarity 0.87)
- MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views (CVPR 2025 · similarity 0.87)
- Volumetric Surfaces: Representing Fuzzy Geometries with Layered Meshes (CVPR 2025 · similarity 0.86)
- GALA: Geometry-Aware Local Adaptive Grids for Detailed 3D Generation (ICLR 2025 · similarity 0.86)
- Geometry Field Splatting with Gaussian Surfels (CVPR 2025 · similarity 0.86)
- GeoSVR: Taming Sparse Voxels for Geometrically Accurate Surface Reconstruction (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.