Research map / Large language models / Language, safety and interpretability
GeoCAD: Local Geometry-Controllable CAD Generation with Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GeoCAD, a method that fine-tunes Llama-3-8B with LoRA to regenerate a masked local loop of a CAD sketch-extrude model from a textual geometric instruction, using a complementary vertex-based and VLLM-based captioning pipeline for ~221k local parts.
Problem
Local geometry-controllable CAD generation: modifying a single local loop of a CAD model so its shape follows user textual geometric instructions while the rest of the model stays unchanged and consistent. Existing methods either ignore text or cannot focus on local parts, and lack fine-grained geom
Approach
Caption local loops with a complementary strategy (geometric rules from vertex coordinates for simple shapes, a VLLM for complex shapes), then fine-tune an LLM with masked-loop infilling where the prompt includes the geometric instruction as a constraint, preceded by an optional CAD-text alignment stage with geometric augmentation.
Key results
- GeoCAD (Llama-3-8B) reaches Ver-score 76.4% and VLLM-score 65.7% versus FlexCAD's 19.8% and 6.93%, and PV 90.5% versus 86.7%.
- Five-shot GeoCAD achieves the best overall numbers: MMD 1.16 (second best), JSD 0.80, PV 92.3%, Ver-score 82.2%, VLLM-score 68.2%, Realism 43.6%.
- Ablation: removing stage 1 pre-training lowers Ver-score from 76.4% to 71.5%; Qwen2.5-7B-Instruct variant reaches Ver-score 79.8%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GeoCAD instead of FlexCAD (five-shot) | Local CAD generation · DeepCAD test set | Ver-score: 43.5% -> 76.4%* | Helped |
| Use GeoCAD instead of FlexCAD (five-shot) | Local CAD generation · DeepCAD test set | VLLM-score: 26.8% -> 65.7%* | Helped |
| Use GeoCAD instead of FlexCAD (five-shot) | Local CAD generation · DeepCAD test set | Realism: 20.2% -> 40.9%* | Helped |
| Add five-shot exemplars | GeoCAD · DeepCAD test set | Ver-score: 76.4%* -> 82.2% | Helped |
| Use complementary vertex-based and VLLM-based captioning instead of vertex-based captioning only (ablation) | GeoCAD · DeepCAD test set | COV: 63.6% -> 64.9% | Helped |
| Use complementary vertex-based and VLLM-based captioning instead of VLLM-based captioning only (ablation) | GeoCAD · DeepCAD test set | COV: 61.8% -> 64.9% | Helped |
| Add stage 1 pre-training for CAD-text alignment (ablation) | GeoCAD · DeepCAD test set | COV: 61.3% -> 64.9% | Helped |
| Add data augmentation during stage 1 pre-training (ablation) | GeoCAD · DeepCAD test set | COV: 62.9% -> 64.9% | Helped |
| Use Qwen2.5-VL-72B-Instruct instead of GPT-4o for complex-part captioning | Complex-part captioning · 1,000 complex local parts | Captioning accuracy: 86.5% -> 91.3% | Helped |
| Use Qwen2.5-7B-Instruct instead of Llama-3-8B (ablation) | GeoCAD · DeepCAD test set | COV: 64.9% -> 68.7% | Helped |
| Use Transformer-4M instead of Llama-3-8B (ablation) | GeoCAD · DeepCAD test set | COV: 64.9% -> 59.1% | Hurt |
| Increase sampling temperature from 0.7 to 1.1 (ablation) | GeoCAD sampling · DeepCAD test set | COV: 63.4% -> 65.6% | Helped |
Surprising findings
- Few-shot prompting of OpenAI-o3 yields weak results (PV 65.7%, Ver-score 33.6%), below fine-tuned open models.
- VLLM captioning accuracy on complex loops is higher for Qwen2.5-VL-72B (91.3%) than GPT-4o (86.5%) on a 1k sample check.
Limitations
- Text-only instructions may not describe some complex loops; hand-drawn image input is future work.
- Generated local parts can conflict structurally with unchanged parts, since DeepCAD lacks inter-part constraint annotations.
- Equilateral triangles do not occur in DeepCAD, limiting shape coverage.
Experimental setup
Datasets: DeepCAD
Benchmarks: DeepCAD test set
Metrics: COV MMD JSD PV Ver-score VLLM-score Realism
Baselines: OpenAI-o3 (five-shot) FlexCAD FlexCAD (five-shot) CAD-Editor
Builds on: FlexCAD DeepCAD LoRA
Closest papers by meaning
- FlexCAD: Unified and Versatile Controllable CAD Generation with Fine-tuned Large Language Models (ICLR 2025 · similarity 0.89)
- CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs (AAAI 2025 · similarity 0.87)
- Text2CAD: Generating Sequential CAD Designs from Beginner-to-Expert Level Text Prompts (NeurIPS 2024 · similarity 0.87)
- CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation (CVPR 2025 · similarity 0.87)
- Generating CAD Code with Vision-Language Models for 3D Designs (ICLR 2025 · similarity 0.87)
- CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.