Research map / Large language models / Language, safety and interpretability
FlexCAD: Unified and Versatile Controllable CAD Generation with Fine-tuned Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FlexCAD, a single LoRA-fine-tuned Llama-3-8B that represents CAD sequences as structured text and uses hierarchy-aware mask prediction to perform controllable CAD generation at all construction hierarchies.
Problem
Controllable CAD generation (sketch-and-extrude models) currently offers limited control across construction hierarchies and requires separate models per control type.
Approach
Convert CAD models into concise structured text with numeric coordinates as decimal integers and hierarchy end tokens, then fine-tune an LLM to infill a masked hierarchy-aware field sampled from multiple prompt templates, enabling one unified model for all hierarchies.
Key results
- Sketch-level controllable generation: FlexCAD reaches 65.6% COV, 1.19 MMD, 0.82 JSD and 93.4% PV, outperforming SkexGen and Hnc-cad on most metrics.
- Extrusion-level: FlexCAD achieves 93.3% PV and 42.1% realism versus 76.1%/35.2% for SkexGen.
- Llama-3-8B with LoRA (0.042% params) matches full fine-tuning performance while converging in 20 hours versus 80 hours.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FlexCAD instead of Hnc-cad | DeepCAD sketch-level controllable generation · DeepCAD test set, sketch-level | PV: 72.6% -> 93.4% | Helped |
| use FlexCAD instead of Hnc-cad | DeepCAD extrusion-level controllable generation · DeepCAD test set, extrusion-level | PV: 79.7%* -> 93.3% | Helped |
| use pretrained weights rather than training Llama-3-8B from scratch (ablation) | FlexCAD · DeepCAD test set, sketch-level | PV: 89.5% -> 93.4% | Helped |
| use LoRA instead of full-parameter fine-tuning (ablation) | Llama-3-8B · DeepCAD test set, sketch-level | inference/training convergence time: 80 hours -> 20 hours | Helped |
| use instruction-tuned Llama-3-8B instead of base Llama-3-8B (ablation) | FlexCAD · DeepCAD test set, sketch-level | PV: 93.4% -> 90.5% | Hurt |
| increase model size from Llama-3-8B to Llama-3-70B (ablation) | FlexCAD · DeepCAD test set, sketch-level | PV: 93.4% -> 94.6% | Helped |
| replace hierarchy-aware masking with random token-span masking (ablation) | FlexCAD · DeepCAD test set, sketch-level | PV: 93.4% -> 90.6% | Hurt |
| replace hierarchy-specific mask tokens with generic [mask] (ablation) | FlexCAD · DeepCAD test set, sketch-level | PV: 93.4% -> 91.5% | Hurt |
| train only sketch-level generation instead of multiple hierarchy tasks (ablation) | FlexCAD · DeepCAD test set, sketch-level | PV: 93.4% -> 92.2% | Hurt |
| represent circles with four points rather than center and radius (ablation) | FlexCAD · DeepCAD test set, sketch-level | PV: 89.7% -> 93.4% | Helped |
| raise sampling temperature from τ = 1.1 to τ = 1.3 (ablation) | FlexCAD inference · DeepCAD test set, sketch-level | PV: 93.4% -> 86.8% | Hurt |
| raise Top-p from 0.9 to 1.0 (ablation) | FlexCAD inference · DeepCAD test set, sketch-level | PV: 93.4% -> 89.0% | Hurt |
Surprising findings
- Llama-3-8B-Full fine-tuning performs similarly to LoRA but needs about 4x longer to converge.
- Unconditional generation JSD is worst for FlexCAD (1.53) versus baselines, though PV and COV are best.
Limitations
- Inference is slower than prior SEM baselines (0.56 s vs 0.15 s and 0.38 s).
- Generates hallucinated irregular polygons when loops are replaced with 5-8 lines, attributed to data imbalance.
- Does not yet support text-to-CAD generation.
Experimental setup
Datasets: DeepCAD
Benchmarks: DeepCAD test set
Metrics: COV MMD JSD Novel Unique PV Realism
Baselines: GPT-4o SkexGen Hnc-cad DeepCAD
Builds on: LoRA DeepCAD SkexGen
Closest papers by meaning
- Text2CAD: Generating Sequential CAD Designs from Beginner-to-Expert Level Text Prompts (NeurIPS 2024 · similarity 0.89)
- GeoCAD: Local Geometry-Controllable CAD Generation with Large Language Models (NeurIPS 2025 · similarity 0.89)
- CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing (ICML 2025 · similarity 0.89)
- CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation (CVPR 2025 · similarity 0.89)
- CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward (NeurIPS 2025 · similarity 0.87)
- Text-to-CAD Generation Through Infusing Visual Feedback in Large Language Models (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.