Research map / Image generation / Text-to-image diffusion
HIFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HIFA, a single-stage NeRF text-to-3D method that combines latent and image-space score distillation, square-root timestep annealing, z-variance regularization, and kernel smoothing for coarse-to-fine importance sampling.
Problem
Text-to-3D generation with NeRFs via score distillation often yields artifacts, cloudy geometry, flickering textures, and inconsistent views, and high-quality results usually require a two-stage (NeRF then mesh) optimization.
Approach
Distill scores in both the latent and image spaces of Stable Diffusion, anneal the sampled diffusion timestep with a square-root schedule, regularize the variance of z-coordinates along rays for crisp geometry, and smooth the coarse-stage PDF with a box kernel for importance sampling, all in a single stage.
Key results
- Qualitative comparisons against DreamFusion, Magic3D, and Fantasia3D show improved photo-realism and texture detail in single-stage optimization (no quantitative numbers given in the text).
- Ablations show square-root timestep annealing outperforms random, linear, and cosine schedules, and z-variance loss removes cloudy artifacts compared to no regularization or distortion loss.
- Image-space loss alone causes color bias, so combining latent and image SDS terms is needed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add image-space supervision to latent-space SDS (ablation) | HIFA · Text-to-3D generation | appearance and texture detail | Helped |
| use image-space loss without latent-space loss (ablation) | HIFA · Text-to-3D generation | color fidelity | Hurt |
| replace square root timestep annealing with random, linear, or cosine sampling (ablation) | HIFA · Text-to-3D generation | appearance and geometry quality | Hurt |
| add z-variance loss (ablation) | HIFA · Text-to-3D generation | geometry quality | Helped |
| replace z-variance loss with distortion loss (ablation) | HIFA · Text-to-3D generation | appearance and geometry quality | Mixed |
| add kernel smoothing to coarse-to-fine importance sampling (ablation) | HIFA · Text-to-3D generation | view consistency | Helped |
Surprising findings
- Image-space SDS loss alone results in color bias regardless of guidance scale.
Limitations
- Experiments with DeepFloyd IF use only the stage-1 model at 64x64 resolution.
- Janus multi-face problem and difficulties with some prompts remain with the SD model.
- Evaluation is primarily qualitative in the provided text.
Experimental setup
Baselines: DreamFusion Magic3D Fantasia3D ProlificDreamer Zero-1-to-3 Magic123 SyncDreamer
Builds on: DreamFusion Stable Diffusion instant-ngp stable-dreamfusion
Closest papers by meaning
- Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation (ICLR 2024 · similarity 0.91)
- GenesisTex: Adapting Image Denoising Diffusion to Texture Space (CVPR 2024 · similarity 0.90)
- DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation (ICLR 2024 · similarity 0.90)
- ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models (CVPR 2024 · similarity 0.90)
- Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data (CVPR 2025 · similarity 0.90)
- Walking the Schrödinger Bridge: A Direct Trajectory for Text-to-3D Generation (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.