Research map / 3D vision / Face and shape
HyperSDFusion: Bridging Hierarchical Structures in Language and Geometry for Enhanced 3D Text2Shape Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HyperSDFusion, a dual-branch latent diffusion model that learns hierarchical text and 3D shape representations in hyperbolic (Poincaré ball) space for text-to-shape generation on Text2Shape.
Problem
Existing text-to-shape methods such as SDFusion ignore the hierarchical structure shared by text (general to detailed descriptions and syntax trees) and 3D shapes (part-to-whole), so generated shapes lack fidelity to detailed prompts.
Approach
Embed text and 3D shape features in hyperbolic space via a hyperbolic text-image encoder (MERU-pretrained), a hyperbolic text-graph convolution over spaCy syntax graphs, a dual-branch 3D U-Net denoiser, and a hyperbolic hierarchical loss that orders deep/middle/shallow shape features by distance to the origin.
Key results
- On 1000 Text2Shape test samples, HyperSDFusion reaches IoU 16.21 and FID 0.70 vs SDFusion's 13.98 and 5.47 (CD 0.7433 vs 1.246).
- On texts longer than 16 words, IoU 13.25 vs SDFusion 10.13 (Text2Shape++ split).
- Hyperbolic hierarchical loss raises the ablation IoU from 15.73 to 16.44 on the mini-set.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HyperSDFusion instead of SDFusion | Text-to-shape generation · Text2Shape test set (random 1000 samples) | IoU: 13.98 -> 16.21 | Helped |
| use HyperSDFusion instead of SDFusion | Text-to-shape generation · Text2Shape test set (random 1000 samples) | CD: 1.246 -> 0.7433 | Helped |
| use HyperSDFusion instead of SDFusion | Text-to-shape generation · Text2Shape test set (random 1000 samples) | F-score: 12.67 -> 15.16 | Helped |
| use HyperSDFusion instead of SDFusion | Text-to-shape generation · Text2Shape test set (random 1000 samples) | FID: 5.47 -> 0.70 | Helped |
| add the hyperbolic text-image encoder (HTIE) (ablation) | Baseline text-to-shape model · Text2Shape mini-set | IoU: 12.74 -> 14.26 | Helped |
| replace the baseline text encoder with T5 (ablation) | Baseline text-to-shape model · Text2Shape mini-set | IoU: 12.74 -> 12.26 | Hurt |
| replace the baseline text encoder with CLIP (ablation) | Baseline text-to-shape model · Text2Shape mini-set | IoU: 12.74 -> 13.99 | Helped |
| use a dual-branch rather than single-branch denoiser (ablation) | HTIE+HTGC diffusion model · Text2Shape mini-set | IoU: 14.18 -> 15.73 | Helped |
| use a dual-branch rather than single-branch denoiser (ablation) | HTIE+HTGC diffusion model · Text2Shape mini-set | F-score: 8.988 -> 9.50 | Helped |
| use hyperbolic text-graph convolution instead of Euclidean GCN (ablation) | HTIE+dual-branch diffusion model · Text2Shape mini-set | IoU: 15.41 -> 15.73 | Helped |
| use hyperbolic hierarchical loss instead of Euclidean hierarchical loss (ablation) | HTIE+HTGC dual-branch diffusion model · Text2Shape mini-set | IoU: 13.51 -> 16.44 | Helped |
| use hyperbolic hierarchical loss instead of Euclidean hierarchical loss (ablation) | HTIE+HTGC dual-branch diffusion model · Text2Shape mini-set | F-score: 9.79 -> 9.917 | Helped |
Surprising findings
- Replacing the text encoder with T5 slightly hurts performance versus the BERT-based baseline (IoU 12.26 vs 12.74).
- The single-branch concatenation variant performs slightly worse than the baseline in IoU and F-score, attributed to feature interference.
Limitations
- Evaluated only on Text2Shape, which covers chairs and tables.
- Hierarchy evaluation relies on self-defined metrics (HMD, HD) and qualitative analysis.
- Hardware and training compute are not reported in the main text.
Experimental setup
Datasets: Text2Shape ShapeNet Text2Shape++
Benchmarks: Text2Shape Text2Shape++
Metrics: IoU Chamfer Distance F-score FID HMD HD
Baselines: SDFusion Liu et al. (AR-based 3D generation) Diffusion-SDF T5 CLIP GCN
Builds on: SDFusion MERU VQ-VAE Hyperbolic graph convolution DDIM
Closest papers by meaning
- PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion (CVPR 2024 · similarity 0.87)
- HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts (NeurIPS 2025 · similarity 0.86)
- Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation (ICLR 2024 · similarity 0.86)
- Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representation (ICLR 2025 · similarity 0.86)
- DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data (CVPR 2024 · similarity 0.86)
- TextField3D: Towards Enhancing Open-Vocabulary 3D Generation with Noisy Text Fields (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.