Research map / 3D vision / 3D generation
Eval3D: Interpretable and Fine-grained Evaluation for 3D Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Eval3D, an open-source, interpretable evaluation framework for text/image-to-3D generation that measures geometric, semantic, structural, aesthetic, and text-3D alignment via consistency among foundation models, and releases a densely annotated 160-prompt benchmark.
Problem
Existing 3D generation evaluation metrics either ignore geometric quality or rely on black-box multimodal LLMs giving coarse, non-localized judgments, and lack multi-view and spatial feedback.
Approach
Treat consistency among foundation-model probes (Depth Anything normals vs. analytic normals, DINOv2 feature variance across views, Zero-123 novel-view prediction with DreamSim, LLM/VLM question answering across views) as a proxy for 3D quality, yielding pixel-wise and 3D-localized scores.
Key results
- Eval3D reaches 88.7% text-3D alignment agreement with humans versus 72.8% for GPT4V, and 83.0% geometric consistency agreement versus 46.9% for GPT4V.
- MVDream ranks first or second on most text-to-3D criteria, but has the lowest-ranked geometric consistency among top models except Gaussian Splatting results show lowest geometric consistency (67.9).
- Replacing LLaVA with GPT-4o for text-3D alignment gives 91.5% human agreement, showing open-source judges are effective.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Eval3D instead of GPT4V for geometric-consistency evaluation | 3D evaluation against human judgments · Human pairwise rating agreement | Geometric consistency agreement: 36% better alignment | Helped |
| Use Eval3D instead of GPT-4V for text-3D alignment evaluation | 3D evaluation against human judgments · Human pairwise rating agreement | Text-3D alignment agreement: approximately 16% better | Helped |
| Use Eval3D rather than prior evaluation metrics | 3D evaluation against human judgments · Human pairwise rating agreement | Agreement across evaluation dimensions | Helped |
| Use image guidance in Magic123 rather than text-only Magic3D | Magic123 · Image-to-3D models | Semantic consistency | Helped |
| Use image guidance in Magic123 rather than text-only Magic3D | Magic123 · Image-to-3D models | Structural consistency | Helped |
| Use image guidance for text+image-to-3D generation | Image-to-3D algorithms compared with text-only algorithms · Image-to-3D models | Text-3D alignment | Helped |
Surprising findings
- ProlificDreamer scores high on aesthetics and text-3D alignment yet ranks last or second-to-last on all three geometric metrics.
- Gaussian Splatting generations rank highly on semantic and structural consistency despite having the lowest geometric consistency.
Limitations
- Accuracy depends on foundation models (Stable-Zero123, Depth Anything, LLaVA), which can introduce errors.
- Evaluated on only six text-to-3D and two image-to-3D methods over 160 prompts.
Experimental setup
Datasets: threestudio prompts GPT-generated prompts from prior works Dreamcraft3D image-text pairs
Benchmarks: Eval3D benchmark
Metrics: geometric consistency semantic consistency structural consistency aesthetics text-3D alignment pairwise agreement with human annotations
Baselines: CLIP-Score ImageReward T3-Bench GPT4V
Builds on: TIFA DSG T3-Bench GPT-4V Elo evaluation
Closest papers by meaning
- Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity (NeurIPS 2025 · similarity 0.92)
- Gen3DEval: Using vLLMs for Automatic Evaluation of Generated 3D Objects (CVPR 2025 · similarity 0.90)
- Interactive3D: Create What You Want by Interactive 3D Generation (CVPR 2024 · similarity 0.87)
- 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code (NeurIPS 2026 · similarity 0.87)
- Let Language Constrain Geometry: Vision–Language Models as Semantic and Spatial Critics for 3D Generation (ICML 2026 · similarity 0.87)
- HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.