Research map / 3D vision / 3D generation
Kiss3DGen: Repurposing Image Diffusion Models for 3D Asset Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Kiss3DGen fine-tunes a Flux DiT with LoRA to generate a 2D "3D Bundle Image" (four RGB views plus normal maps), then reconstructs textured meshes via ISOMER/LRM and extends to enhancement and editing with ControlNet.
Problem
3D asset generation is limited by scarce high-quality 3D training data, and existing direct 3D generators have weak generalization while optimization-based methods are slow.
Approach
Repack multi-view RGB images and their normal maps into a single tiled image so 3D generation becomes a 2D image generation task, letting a pretrained image DiT (Flux) be retargeted with LoRA; ControlNet on the same bundle format enables enhancement and editing.
Key results
- Kiss3DGen-Base reaches CLIP 0.837 and Quality 2.700 in text-to-3D, beating 3DTopia, Direct2.5 and Hunyuan3D-1.0 on all listed metrics.
- In image-to-3D on GSO, Ours-Base achieves Chamfer Distance 0.149 and F-Score 0.769, versus 0.153/0.768 for Hunyuan3D-1.0.
- Ours-Base obtains PSNR 20.348 and LPIPS 0.116 versus 19.237 and 0.127 for Unique3D; a 50k-object variant remains competitive.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Kiss3DGen-Base instead of MVDream | Text-to-multi-view synthesis · Text-to-multi-view synthesis | CLIP: 0.809 -> 0.844 | Helped |
| Use Kiss3DGen-Base instead of Hunyuan3D-1.0 | Text-to-3D generation · Text-to-3D generation | CLIP: 0.792 -> 0.837 | Helped |
| Use Kiss3DGen-Base instead of Hunyuan3D-1.0 | Image-to-3D generation · Image-to-3D generation | PSNR: 16.652 -> 20.348 | Helped |
| Train on 50K objects instead of 147K (ablation) | Kiss3DGen-Base · Text-to-3D generation | CLIP: 0.837 -> 0.804 | Hurt |
| Train on 50K objects instead of 147K (ablation) | Kiss3DGen-Base · Text-to-3D generation | Quality: 2.700 -> 2.716 | Helped |
| Train on 50K objects instead of 147K (ablation) | Kiss3DGen-Base · Image-to-3D generation | SSIM: 0.902 -> 0.884 | Hurt |
| Train on 50K objects instead of 147K (ablation) | Kiss3DGen-Base · Image-to-3D generation | PSNR: 20.348 -> 20.215 | Hurt |
Surprising findings
- Generated multi-view images outscore the GPT-4V-annotated real data on Q-Align quality and aesthetic metrics, attributed to inheritance of Flux's high-quality image priors.
Limitations
- Model trained on small data (50k) sometimes fails to generate a 3D Bundle Image.
- Relies on optimization-based mesh refinement (ISOMER) and LRM initialization, adding inference time.
- Future work noted on optimal geometry representation and high-resolution view generation.
Experimental setup
Datasets: Objaverse GSO cartoon human body dataset (4k)
Benchmarks: GSO
Metrics: CLIP score Q-Align quality Q-Align aesthetic Chamfer Distance F-Score PSNR SSIM LPIPS
Baselines: MVDream 3DTopia Direct2.5 Hunyuan3D-1.0 Unique3D CraftsMan MVEdit
Builds on: Flux ControlNet ISOMER LoRA InstantMesh
Closest papers by meaning
- One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion (CVPR 2024 · similarity 0.91)
- Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle (AAAI 2025 · similarity 0.90)
- Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priors (CVPR 2024 · similarity 0.89)
- Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape (ICLR 2024 · similarity 0.89)
- DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior (ICLR 2024 · similarity 0.88)
- Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.