Research map / 3D vision / 3D generation
Scalable 3D Captioning with Pretrained Models
Tiange Luo, Chris Rockwell, Honglak Lee, Justin Johnson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace crowdsourced annotation with Cap3D | Objaverse captioning · Objaverse, 22k objects | A/B win rate: 37.8% ± 0.5% -> 52.3% ± 0.5% | Improved |
| Replace crowdsourced annotation with Cap3D | Objaverse captioning · Objaverse, 22k objects | annotation cost per 1k objects: $87.18 -> $8.35 | Improved |
| Replace crowdsourced annotation with Cap3D | Objaverse captioning · Objaverse, 22k objects | annotation speed: 1.4k / day -> 65k / day | Improved |
| Add GPT4 consolidation over multiple rendered views (ablation) | BLIP2 captioning · Objaverse, full-dataset random 5k split | ViLT image retrieval R@5: 24.7 -> 31.9 | Improved |
| Add CLIP filtering before GPT4 consolidation (ablation) | BLIP2 + GPT4 captioning · Objaverse, full-dataset random 5k split | CLIPScore: 86.3 -> 86.9 | Improved |
| Add CLIP filtering before GPT4 consolidation (ablation) | BLIP2 + GPT4 captioning · Objaverse, full-dataset random 5k split | ViLT image retrieval R@5: 31.9 -> 31.1 | Mixed |
| Use two-stage geometry question answering instead of ordinary captioning (ablation) | Cap3D on ABO · ABO, 6.4k objects | A/B win rate against Human: 47.3±1% -> 50.2±1% | Improved |
| Use two-stage geometry question answering instead of ordinary captioning (ablation) | Cap3D on ABO · ABO, 6.4k objects | A/B win rate against ordinary Cap3D: 56.0±1% win; 37.4±1% lose | Improved |
| Finetune Point·E on 30k Cap3D captions rather than 30k human captions | Point·E · Objaverse text-to-3D | FID: 34.6 -> 33.7 | Improved |
| Increase Point·E finetuning data from 30k to 350k Cap3D captions | Point·E · Objaverse text-to-3D | FID: 33.7 -> 32.8 | Improved |
| Finetune Shap·E STF on 350k Cap3D captions | Shap·E STF · Objaverse text-to-3D | CLIPScore: 80.4 -> 79.1 | Hurt |
| Finetune Shap·E STF on 350k Cap3D captions | Shap·E STF · Objaverse text-to-3D | FID: 37.2 -> 35.5 | Improved |
Closest papers by meaning
- MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creation (CVPR 2025 · similarity 0.90)
- Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts (ICLR 2024 · similarity 0.87)
- CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Models (ICML 2026 · similarity 0.87)
- SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE (CVPR 2025 · similarity 0.87)
- More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding (AAAI 2025 · similarity 0.86)
- ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
