Research map / Vision-language and multimodal / Human and embodied modeling
Text2CAD: Generating Sequential CAD Designs from Beginner-to-Expert Level Text Prompts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Text2CAD, a text-to-parametric-CAD framework with a multi-level text annotation pipeline over DeepCAD (LLM/VLM-generated prompts) and a transformer that autoregressively generates sketch-and-extrude sequences from text.
Problem
Generating parametric CAD construction sequences (sketch and extrusion) from text prompts of varying detail is unsupported, mainly because no dataset pairs CAD design histories with natural language instructions.
Approach
A two-stage annotation pipeline (LLaVA-NeXT shape descriptions, then Mixtral LLM multi-level instructions from minimal metadata) creates four prompt levels per DeepCAD model, and an end-to-end autoregressive transformer with BERT text encoder and layer-wise cross-attention maps text directly to CAD tokens.
Key results
- On expert (L3) prompts, Text2CAD reaches median CD 0.37 vs 32.82 for DeepCAD (x10^-3) and invalidity ratio 0.93% vs 10.00%.
- GPT-4V preference for Text2CAD over DeepCAD is 63.24% at expert level and 51.80% at abstract level.
- Arc F1 rises from 20.04 (DeepCAD) to 36.03 (Text2CAD) on expert prompts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Text2CAD instead of DeepCAD | Text-to-CAD generation · DeepCAD test, expert prompts (L3) | Arc F1: 20.04 -> 36.03 | Helped |
| Use Text2CAD instead of DeepCAD | Text-to-CAD generation · DeepCAD test, expert prompts (L3) | Median CD (×10³): 32.82 -> 0.37 | Helped |
| Use Text2CAD instead of DeepCAD | Text-to-CAD generation · DeepCAD test, expert prompts (L3) | Invalidity ratio: 10.00 -> 0.93 | Helped |
| Use Text2CAD instead of DeepCAD | Text-to-CAD generation · DeepCAD test, beginner prompts (L1) | GPT-4V preference (%): 51.15 -> 48.35 | Hurt |
| Use Text2CAD instead of DeepCAD | Text-to-CAD generation · DeepCAD test, intermediate prompts (L2) | GPT-4V preference (%): 40.20 -> 58.80 | Helped |
| Use Text2CAD instead of DeepCAD | Text-to-CAD generation · DeepCAD test, expert prompts (L3) | GPT-4V preference (%): 36.06 -> 63.24 | Helped |
| Use Text2CAD instead of DeepCAD | Text-to-CAD generation · DeepCAD test, abstract prompts (L0) | User preference (%): 50.95 -> 49.05 | Hurt |
| Use Text2CAD instead of DeepCAD | Text-to-CAD generation · DeepCAD test, expert prompts (L3) | User preference (%): 41.14 -> 58.86 | Helped |
| Add the Adaptive Layer (ablation) | Text2CAD · DeepCAD test, expert prompts (L3) | Arc F1: 27.18 -> 36.03 | Helped |
| Add the Adaptive Layer (ablation) | Text2CAD · DeepCAD test, expert prompts (L3) | Invalidity ratio: 2.69 -> 0.93 | Helped |
Surprising findings
- Text2CAD slightly loses to DeepCAD at the beginner (L1) level in GPT-4V evaluation (48.35% vs 51.15%), with gaps near zero for abstract prompts.
Limitations
- LLaVA-NeXT shape descriptions are sensitive to perspective distortion in multi-view images, propagating noise into prompts.
- No standardized benchmark exists for text-to-CAD evaluation.
- DeepCAD is imbalanced toward rectangular and cylindrical shapes, limiting robustness on complex shapes.
Experimental setup
Datasets: DeepCAD
Benchmarks: DeepCAD test set
Metrics: F1 (line, arc, circle, extrusion) median chamfer distance mean chamfer distance invalidity ratio GPT-4V preference user study preference
Baselines: DeepCAD Text2CAD w/o Adaptive Layer
Builds on: DeepCAD BERT LLaVA-NeXT
Closest papers by meaning
- CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing (ICML 2025 · similarity 0.90)
- Text-to-CAD Evaluation with CADTests (NeurIPS 2026 · similarity 0.89)
- FlexCAD: Unified and Versatile Controllable CAD Generation with Fine-tuned Large Language Models (ICLR 2025 · similarity 0.89)
- Clarify Before You Draw: Proactive Agents for Robust Text-to-CAD Generation (ICML 2026 · similarity 0.89)
- From 2D CAD Drawings to 3D Parametric Models: A Vision-Language Approach (AAAI 2025 · similarity 0.88)
- CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.