Research map / 3D vision / Face and shape
SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents SF3D, a TripoSR-based feed-forward model that turns a single image into a UV-unwrapped, textured, delit mesh with predicted metallic/roughness and smoothed DMTet geometry in about 0.3-0.5 s.
Problem
Feed-forward single-image 3D reconstruction produces meshes that are hard to use downstream: vertex colors instead of textures, baked-in lighting, marching-cubes stair-step artifacts, and no material properties. Slow UV unwrapping makes fixing the vertex-color issue costly.
Approach
Extends TripoSR with a higher-resolution triplane transformer (DINOv2 features, 384x384 triplanes), a Beta-distribution material network for metallic/roughness, a spherical-Gaussian light net for illumination disentanglement, DMTet with learned vertex offsets and normals, and a parallelizable cube-projection UV unwrapping that runs in about 10 ms.
Key results
- On GSO, SF3D reaches CD 0.096 and FS@0.1 0.710 versus TripoSR's 0.111 and 0.645, with PSNR 17.619 vs 16.445.
- On OmniObject3D, SF3D achieves CD 0.089 and PSNR 15.136, outperforming all listed baselines including InstantMesh (CD 0.138, PSNR 13.531).
- SF3D runs in about 0.3 s per image on an H100, on par with TripoSR and much faster than InstantMesh (32.4 s) and LGM (64.6 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add mesh-based training, relighting fine-tuning, and vertex offsets to TripoSR (ablation) | SF3D without Enhanced Transformer · GSO | Chamfer Distance: 0.111 -> 0.108 | Helped |
| add the enhanced transformer for higher-resolution triplanes (ablation) | SF3D · GSO | Chamfer Distance: 0.108 -> 0.098 | Helped |
| add the enhanced transformer for higher-resolution triplanes (ablation) | SF3D · GSO | FS@0.1: 0.660 -> 0.701 | Helped |
| add the enhanced transformer for higher-resolution triplanes (ablation) | SF3D · GSO | FS@0.2: 0.872 -> 0.894 | Helped |
| add the enhanced transformer for higher-resolution triplanes (ablation) | SF3D · GSO | FS@0.5: 0.982 -> 0.988 | Helped |
| replace TripoSR with SF3D | single-image 3D reconstruction · GSO | PSNR: 16.445 -> 17.619 | Helped |
| replace TripoSR with SF3D | single-image 3D reconstruction · OmniObject3D | Chamfer Distance: 0.103 -> 0.089 | Helped |
| replace TripoSR with SF3D | single-image 3D reconstruction · OmniObject3D | LPIPS: 0.224 -> 0.209 | Helped |
Surprising findings
- Directly regressing material values led to training collapse, with the network always predicting roughness 0.5 and metallic 0; a Beta-distribution likelihood fixed this.
Limitations
- Material roughness and metallic values are homogeneous over the object, so spatially varying multi-material objects are not handled.
- Input is a single LDR image, so dark regions lack information and albedo can be mismatched (e.g., the cup example).
- Material prediction and delighting are learned without explicit supervision, so the approach relies on synthetic training data.
Experimental setup
Datasets: synthetic multi-view training dataset (Objaverse-style, unnamed in text) GSO OmniObject3D Polyhaven
Benchmarks: GSO OmniObject3D
Metrics: Chamfer Distance F-Score@0.1 F-Score@0.2 F-Score@0.5 PSNR SSIM LPIPS inference time
Baselines: TripoSR OpenLRM LGM CRM InstantMesh ZeroShape 3DTopia-XL LN3Diff
Builds on: TripoSR PointInfinity DMTet NeRD MeshLRM
Closest papers by meaning
- One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion (CVPR 2024 · similarity 0.87)
- Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models (CVPR 2024 · similarity 0.86)
- ARM: Appearance Reconstruction Model for Relightable 3D Generation (CVPR 2025 · similarity 0.86)
- MeshGen: Generating PBR Textured Mesh with Render-Enhanced Auto-Encoder and Generative Data Augmentation (CVPR 2025 · similarity 0.86)
- Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image (NeurIPS 2024 · similarity 0.86)
- GTR: Improving Large 3D Reconstruction Models through Geometry and Texture Refinement (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.