Research map / 3D vision / Face and shape
MeshFormer : High-Quality Mesh Generation with 3D-Guided Reconstruction Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MeshFormer, a sparse-view feed-forward reconstruction model that uses 3D sparse voxels with transformer-plus-3D-convolution, multi-view normal inputs, and SDF-supervised surface rendering to produce high-quality textured meshes in a single stage.
Problem
Feed-forward open-world sparse-view 3D reconstruction is expensive to train and triplane/NeRF-based pipelines struggle to extract high-quality meshes with fine geometric detail.
Approach
Replace triplanes with explicit 3D sparse voxel features processed by a UNet with a transformer bottleneck and projection-aware cross-attention; feed multi-view normal maps as input; train end-to-end on mesh surface rendering plus high-resolution SDF supervision, with a learned normal texture used for post-hoc geometry enhancement.
Key results
- On GSO single-image-to-3D, MeshFormer reaches F-score 0.963 and CD 0.031, versus 0.956 and 0.033 for the next best (MeshLRM).
- On OmniObject3D, MeshFormer reaches F-score 0.914, CD 0.043, PSNR 18.14, beating MeshLRM (0.910, 0.045, 18.10).
- Under limited resources (8 H100 for 48h), MeshLRM's GSO F-score drops to 0.925 while MeshFormer reaches 0.960.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add multi-view normal input (ablation) | MeshFormer · GSO | CD: 0.024 -> 0.012 | Helped |
| Add SDF supervision (ablation) | MeshFormer · GSO | CD: 0.035 -> 0.012 | Helped |
| Add transformer layers (ablation) | MeshFormer · GSO | LPIPS-C: 0.101 -> 0.083 | Helped |
| Use projection-aware cross-attention instead of average pooling (ablation) | MeshFormer · GSO | CD: 0.013 -> 0.012 | Helped |
| Add geometry enhancement (ablation) | MeshFormer · GSO | CD: 0.012 -> 0.012 | No clear effect |
| Replace ground-truth input normals with predicted normals (ablation) | MeshFormer · GSO | F-Score: 0.992 -> 0.987 | Hurt |
| Train with MeshFormer instead of MeshLRM under 8×H100 48h | MeshFormer and MeshLRM · GSO | F-Score: 0.925 -> 0.960 | Helped |
| Increase training from 12h to 168h (ablation) | MeshFormer · GSO | F-Score: 0.960 -> 0.963 | Helped |
Surprising findings
- Removing the SDF supervision after the first epoch causes geometry learning to deteriorate quickly, with PSNR-C falling from 28.15 to 20.72 in the ablation.
- Replacing ground-truth input normals with Zero123++-predicted normals at test time causes a notable drop in color and normal quality relative to the full model.
Limitations
- Relies on 2D diffusion models (Zero123++) for multi-view RGB and normal inputs, whose inconsistency can degrade performance.
- Predicted normal maps have errors relative to ground truth, causing a gap versus GT normal inputs.
Experimental setup
Datasets: Objaverse GSO OmniObject3D
Benchmarks: GSO OmniObject3D
Metrics: F-score Chamfer distance PSNR LPIPS
Baselines: One-2-3-45++ TripoSR CRM LGM InstantMesh MeshLRM
Builds on: Large Reconstruction Model (LRM) Zero123++ DINOv2 NVDiffRast Dual Marching Cubes
Closest papers by meaning
- Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modeling (NeurIPS 2025 · similarity 0.89)
- Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attention (NeurIPS 2025 · similarity 0.89)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.89)
- Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors (ICLR 2024 · similarity 0.88)
- Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views (AAAI 2024 · similarity 0.88)
- One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.