Research map / Vision-language and multimodal / Human and embodied modeling
From 2D CAD Drawings to 3D Parametric Models: A Vision-Language Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reconstructs 3D cabinet parametric models from raw 2D CAD drawings by feeding raster images into a fine-tuned vision-language model that outputs a Python-style shape program.
Problem
Reconstructing 3D parametric CAD models from 2D engineering drawings is hard for existing methods, which need clean vector geometry, exactly three orthographic views, and fixed-slot sequence representations that do not scale to many primitives or model-specific parameters.
Approach
Treat the whole 2D drawing (geometry and annotation layers) as a raster image encoded by a ViT, and auto-regressively generate a general-purpose Python text program describing primitive IDs, box parameters, and model-specific parameters, fine-tuning a open VLM end to end.
Key results
- Mini-InternVL-1.5-2B with both geometry and annotation layers reaches 93.80% retrieval accuracy, 82.76 F1 and 97.21% parameter accuracy, versus 62.65 F1 with geometry only.
- The text-based Python shape program matches a command-template variant on retrieval and reconstruction (F1 82.65 vs 82.07 for command format) while also supporting model-specific parameters.
- Swapping the VLM to QWen2-VL-2B gives 84.90 F1 and 93.98% retrieval accuracy, slightly better on box reconstruction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Include the drawing's annotation layer (ablation) | CAD2Program · Cabinet test set | reconstruction F1: 62.65 -> 82.76 | Helped |
| Include the drawing's annotation layer (ablation) | CAD2Program · Cabinet test set | retrieval accuracy: 85.42 -> 93.80 | Helped |
| Include the drawing's annotation layer (ablation) | CAD2Program · Cabinet test set | parameter accuracy: 81.94 -> 97.21 | Helped |
| Use text rather than a domain-specific command template | CAD2Program · Cabinet test set | reconstruction F1: 82.07 -> 82.76 | Helped |
| Add model-specific parameters to the Python output (ablation) | CAD2Program · Cabinet test set | parameter accuracy: n/a -> 97.21 | Helped |
| Replace Python with YAML as the proxy language (ablation) | CAD2Program · Cabinet test set | reconstruction F1: 82.76 -> 83.05 | No clear effect |
| Replace Mini-InternVL-1.5-2B with QWen2-VL-2B (ablation) | CAD2Program · Cabinet test set | reconstruction F1: 82.76 -> 84.90 | Helped |
Surprising findings
- Including the annotation layer (dimension sets) substantially improves reconstruction, contrary to traditional practice of removing annotations.
- A ViT raster-image encoder on PlankAssembly performs comparably to the vector-input model across noise levels.
Limitations
- Evaluated only on cabinet furniture
- Predicted 3D positions sometimes inaccurate, leaving gaps between parts
- Coordinate system ambiguity causes pose mismatches for L-shaped cabinets
Experimental setup
Datasets: 364K cabinet models with engineering drawings from an online interior design platform PlankAssembly dataset
Benchmarks: in-house cabinet dataset (2K test split) PlankAssembly dataset
Metrics: model retrieval accuracy precision recall F1 (3D IoU > 0.5) parameter estimation accuracy
Baselines: PlankAssembly (Vector) PlankAssembly (ViT) Command-template sequence model (encoder-decoder with 6-layer Transformer decode YAML proxy language variant
Builds on: InternVL PlankAssembly DeepCAD SWIFT
Closest papers by meaning
- Text-to-CAD Generation Through Infusing Visual Feedback in Large Language Models (ICML 2025 · similarity 0.90)
- CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation (CVPR 2025 · similarity 0.90)
- Text2CAD: Generating Sequential CAD Designs from Beginner-to-Expert Level Text Prompts (NeurIPS 2024 · similarity 0.88)
- CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images (CVPR 2025 · similarity 0.88)
- MiCADangelo: Fine-Grained Reconstruction of Constrained CAD Models from 3D Scans (NeurIPS 2025 · similarity 0.88)
- DrawingsDreamer: A Unified Multi-View Engineering Drawings Generation Model (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.