Research map / 3D vision / 3D generation
Generating CAD Code with Vision-Language Models for 3D Designs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CADCodeVerify, a VLM self-verification loop that generates and answers yes/no validation questions about rendered CAD objects to refine CadQuery code, and a 200-prompt CADPrompt benchmark.
Problem
LLM/VLM-generated CAD scripting code often produces 3D objects that do not meet the user's specifications, and existing refinement methods rely on human-in-the-loop feedback.
Approach
An automated refinement loop where the VLM generates binary verification questions from the prompt, answers them against reference images of the rendered object (four viewpoints), and converts the Q&A into feedback to correct the CadQuery code, without human involvement or external geometric solvers.
Key results
- On GPT-4 zero-shot, CADCodeVerify reduces median Point Cloud distance from 0.153 to 0.132 (Refine stage best) and raises compile rate from 92.0% to 94.0%, versus 3D-Premise at 0.150.
- On Gemini zero-shot, CADCodeVerify achieves 0.138 Point Cloud distance vs 0.150 for 3D-Premise, a reduction of over 7%.
- GPT-4 few-shot achieves 96.5% compile rate on CADPrompt, higher than Gemini (85%) and CodeLlama (73.5%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply CADCodeVerify instead of 3D-Premise | GPT-4 few-shot CAD code generation · CADPrompt | Point Cloud distance: 0.137 -> 0.127 | Helped |
| apply CADCodeVerify instead of 3D-Premise | Gemini few-shot CAD code generation · CADPrompt | Point Cloud distance: 0.169 -> 0.147 | Helped |
| apply CADCodeVerify instead of the unrefined generation | CodeLlama few-shot CAD code generation · CADPrompt | Compile rate: 67.0% -> 73.5% | Helped |
| apply CADCodeVerify instead of 3D-Premise | GPT-4 few-shot CAD code generation · CADPrompt | Compile rate: 91.0% -> 96.5% | Helped |
| add reference images during refinement (ablation) | CADCodeVerify with GPT-4 few-shot · CADPrompt, 100 examples | Point Cloud distance: 0.153 -> 0.126 | Helped |
| add reference images during refinement (ablation) | CADCodeVerify with GPT-4 few-shot · CADPrompt, 100 examples | Hausdorff distance: 0.451 -> 0.444 | Helped |
| add few-shot question examples (ablation) | CADCodeVerify with GPT-4 few-shot · CADPrompt, 100 examples | Point Cloud distance: 0.141 -> 0.126 | Helped |
| add few-shot question examples (ablation) | CADCodeVerify with GPT-4 few-shot · CADPrompt, 100 examples | Hausdorff distance: 0.471 -> 0.444 | Helped |
| use CADCodeVerify rather than 3D-Premise | GPT-4 on complex CADPrompt objects · CADPrompt Complex split | Compile rate: 87.4% -> 96.6% | Helped |
Surprising findings
- The 3D-Premise feedback reduced GPT-4's compile rate on Hard and Complex splits (e.g., a 20% drop on Hard at Refine-1).
- Human-in-the-loop feedback only slightly outperforms CADCodeVerify (Point Cloud 0.120 vs 0.137), and CADCodeVerify's answer accuracy was only about 65-68% including Unclear responses.
Limitations
- Point Cloud and Hausdorff distances are noisy and do not capture structural or logical errors.
- Results depend on user prompt phrasing, and multiple valid prompts exist per object.
- Answer accuracy of VLM verification is limited (about 65-68% correct including Unclear), and the approach relies on GPT-4 for multimodal feedback.
Experimental setup
Datasets: CADPrompt CADQuery documentation code examples (40 few-shot examples)
Benchmarks: CADPrompt
Metrics: Point Cloud distance Hausdorff distance IoGT compile rate
Baselines: 3D-Premise Geometric solver feedback Generated (no refinement) CodeLlama
Builds on: 3D-Premise Chain-of-Thought
Closest papers by meaning
- CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation (CVPR 2025 · similarity 0.89)
- Text-to-CAD Generation Through Infusing Visual Feedback in Large Language Models (ICML 2025 · similarity 0.89)
- CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward (NeurIPS 2025 · similarity 0.89)
- 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code (NeurIPS 2026 · similarity 0.89)
- From 2D CAD Drawings to 3D Parametric Models: A Vision-Language Approach (AAAI 2025 · similarity 0.87)
- Text-to-CAD Evaluation with CADTests (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.