Research map / 3D vision / 3D generation
3DHumanEdit: Multi-modal Body Part-aware Conditioning Information Integration for 3D Human Manipulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3DHumanEdit, a body part-aware text and segmentation-conditioned editing method for 3D GAN-generated humans that fuses T5 and CLIP text features with segmentation features, adds a pose/shape estimator, and enforces multi-view consistency.
Problem
Fine-grained, multi-view consistent manipulation of individual body parts in 3D GAN-generated human images based on text and segmentation inputs.
Approach
Combines T5-encoded long text with CLIP part-level token features via cross-modal attention between segmentation images and text, aligns text tokens to body-part masks with an attention loss, and adds a pose-and-shape estimator (SMPL parameters) plus a multi-view LPIPS/L2 consistency loss on positive and negative yaw views.
Key results
- FID of 11.4632 versus 13.8122 for Text2Human and 15.0332 for CCH (best).
- CLIP-S(L) of 26.1144 versus 24.5152 for CCH, the strongest baseline.
- Ablations show removing the shape estimator drops CLIP-S(B) from 31.31 to 26.19.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 3DHumanEdit instead of CCH | 3D human text-guided manipulation · DeepFashion test | FID: 15.0332 -> 11.4632 | Helped |
| Use 3DHumanEdit instead of CCH | 3D human text-guided manipulation · DeepFashion test | KID: 0.0036 -> 0.0029 | Helped |
| Use 3DHumanEdit instead of CCH | 3D human text-guided manipulation · DeepFashion test | CLIP-S(L): 24.5152 -> 26.1144 | Helped |
| Add shape information (ablation) | 3DHumanEdit · DeepFashion test | CLIP-S(B): 26.1881 -> 31.3093 | Helped |
| Add shape information (ablation) | 3DHumanEdit · DeepFashion test | CLIP-S(L): 22.0422 -> 26.1144 | Helped |
| Add shape information (ablation) | 3DHumanEdit · DeepFashion test | CLIP-S(G): 22.8471 -> 25.1882 | Helped |
| Add token-correlated features (ablation) | 3DHumanEdit · DeepFashion test | CLIP-S(B): 27.3537 -> 31.3093 | Helped |
| Add token-correlated features (ablation) | 3DHumanEdit · DeepFashion test | CLIP-S(L): 23.6756 -> 26.1144 | Helped |
| Add token-correlated features (ablation) | 3DHumanEdit · DeepFashion test | CLIP-S(G): 23.0038 -> 25.1882 | Helped |
| Add the T5 encoder (ablation) | 3DHumanEdit · DeepFashion test | CLIP-S(B): 28.8767 -> 31.3093 | Helped |
| Add the T5 encoder (ablation) | 3DHumanEdit · DeepFashion test | CLIP-S(L): 24.0788 -> 26.1144 | Helped |
| Add the T5 encoder (ablation) | 3DHumanEdit · DeepFashion test | CLIP-S(G): 23.8984 -> 25.1882 | Helped |
Surprising findings
- Removing the T5 encoder slightly improved some CLIP-Score values (CLIP-S(B) 28.88 vs 31.31 full model is lower, but CLIP-S(L) 24.08 vs 26.11 ), which the authors note as a partial exception.
Limitations
- Multi-view consistency is assessed mainly qualitatively; no quantitative 3D consistency metric is reported.
- Evaluation relies on a single in-house synthetic dataset generated by another 3D generator, limiting external validity.
Experimental setup
Datasets: DeepFashion
Benchmarks: 10,000 randomly selected text descriptions (in-house generated set)
Metrics: FID KID CLIP-Score
Baselines: Text2Human DreamAvatar Text2Avatar CCH
Builds on: EVA3D CLIP T5 SMPL Text2Human
Closest papers by meaning
- AttriHuman-3D: Editable 3D Human Avatar Generation with Attribute Decomposition and Indexing (CVPR 2024 · similarity 0.87)
- SyncHuman: Synchronizing 2D and 3D Generative Models for Single-view Human Reconstruction (NeurIPS 2025 · similarity 0.85)
- En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data (CVPR 2024 · similarity 0.85)
- SCULPT: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes (CVPR 2024 · similarity 0.84)
- IAAO: Interactive Affordance Learning for Articulated Objects in 3D Environments (CVPR 2025 · similarity 0.84)
- Reference-Based 3D-Aware Image Editing with Triplanes (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.