Research map / 3D vision / 3D generation
View Selection for 3D Captioning via Diffusion Ranking
Tiange Luo, Justin Johnson, Honglak Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select top-6 views with DiffuRank instead of the bottom-6 views for GPT4-Vision captioning (ablation) | DiffuRank + GPT4-Vision captioning · Objaverse captions, quality A/B test | quality score (1-5): 2.74 -> lower is better for the other caption; bottom-6 views lose to ours with 31.1 win / 52.0 lose | Improved |
| use DiffuRank top-6 views instead of all 28 views (ablation) | GPT4-Vision captioning · Objaverse captions, CLIP score | CLIP score: 73.5 -> 74.6 | Improved |
| use DiffuRank top-6 views instead of horizontal 6 views (ablation) | GPT4-Vision captioning · Objaverse captions, CLIP R@1 | CLIP R@1: 25.8 -> 26.7 | Improved |
| use DiffuRank top-6 views instead of bottom-6 views (ablation) | GPT4-Vision captioning · Objaverse captions, hallucination A/B test | hallucination score (1-5): 2.61 vs 2.85 (all views) and 2.90 (horizontal); bottom-6 loses 57.0% | Improved |
| replace Cap3D captions with DiffuRank + GPT4-Vision captions | Objaverse captions · Objaverse captions (5k objects) | CLIP R@1: 20.5 -> 26.7 | Improved |
| fine-tune Shap-E (STF) on Cap3D captions | Shap-E (STF) · Text-to-3D finetuning | CLIP R@1: 12.7 -> 11.9 | Hurt |
| fine-tune Shap-E (NeRF) on Cap3D captions | Shap-E (NeRF) · Text-to-3D finetuning | CLIP R@1: 12.2 -> 11.7 | Hurt |
| use corrected captions (330k, ~20% updated) instead of Cap3D captions | Shap-E (STF) · Text-to-3D finetuning | CLIP R@1: 11.9 -> 13.4 | Improved |
| scale training captions from 330k to 825k (ablation) | Shap-E (STF) · Text-to-3D finetuning | CLIP R@1: 13.4 -> 14.9 | Improved |
| scale training captions from 330k to 825k (ablation) | Point-E · Text-to-3D finetuning | CLIP R@1: 8.1 -> 10.1 | Improved |
| fine-tune Point-E on Cap3D captions | Point-E · Text-to-3D finetuning | CLIP R@1: 3.4 -> 7.1 | Improved |
Closest papers by meaning
- Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation (ICLR 2024 · similarity 0.87)
- MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations (NeurIPS 2024 · similarity 0.87)
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.87)
- Visual Fact Checker: Enabling High-Fidelity Detailed Caption Generation (CVPR 2024 · similarity 0.87)
- MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creation (CVPR 2025 · similarity 0.86)
- DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
