Research map / 3D vision / 3D generation
CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a depth/normal-conditioned latent diffusion transformer on synthetic DeepCAD renderings to generate CAD command sequences from unconstrained images, refined with DPO using a CAD compiler as a validity checker.
Problem
Reverse engineering parametric CAD command sequences from unconstrained real-world images, where real paired CAD-image data is scarce and models trained on synthetic data generalize poorly.
Approach
Condition a latent diffusion transformer on texture-invariant depth and normal features (via DINOv2 and a multi-view transformer geometry encoder), distill the multi-view encoder into a single-view encoder, and fine-tune with DPO where positive/negative samples are determined by whether the decoded CAD sequence compiles.
Key results
- On the RealCAD real-world dataset, CADCrafter (multi-view) achieves 83.18% command accuracy, 66.89% parameter accuracy, 0.062 Median CD and 0.067 invalid rate, outperforming DeepCAD and HNC-CAD retrained baselines.
- On the synthetic DeepCAD test set, CADCrafter (multi-view) reaches Median CD 0.026 and invalid rate 0.036 versus 0.113 and 0.106 for DeepCAD multi-view.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CADCrafter instead of Img2CAD for image-to-CAD generation | CADCrafter_s · DeepCAD test set (synthetic) | Acccmd (%): 80.57 -> 83.23 | Helped |
| Use CADCrafter instead of Wonder3D† for reconstruction | CADCrafter_s · DeepCAD test set (synthetic) | Med CD: 0.133 -> 0.049 | Helped |
| Use CADCrafter instead of DeepCAD*_s on real-world images | CADCrafter_s · RealCAD | Acccmd (%): 56.59 -> 81.23 | Helped |
| Add the geometry encoder rather than using concatenated DINO features and an MLP (ablation) | CADCrafter · DeepCAD, single-view | Med CD: 0.056 -> 0.049 | Helped |
| Use multi-view pretraining rather than training from scratch (ablation) | CADCrafter single-view · DeepCAD, single-view | Med CD: 0.079 -> 0.049 | Helped |
| Add the multi-view-to-single-view distillation loss (ablation) | CADCrafter · DeepCAD, single-view | Med CD: 0.068 -> 0.049 | Helped |
| Add DPO fine-tuning with the CAD compiler code checker (ablation) | CADCrafter · DeepCAD, single-view | IR: 0.081 -> 0.042 | Helped |
| Use RGB+Depth+Normal instead of Depth+Normal (ablation) | CADCrafter multi-view · DeepCAD test set (synthetic) | Acccmd (%): 84.62 -> 85.18 | Helped |
| Use RGB+Depth+Normal instead of Depth+Normal (ablation) | CADCrafter multi-view · RealCAD | Med CD: 0.062 -> 0.192 | Hurt |
Surprising findings
- Adding RGB input improves synthetic accuracy slightly but markedly hurts generalization to real images, so RGB was excluded.
- Image-to-3D methods (TripoSR, Wonder3D, One-2-3-45) have Median CD on RealCAD that is competitive with or better than some CAD baselines, though CADCrafter still outperforms them.
Limitations
- Trained only on mechanical parts from DeepCAD, so complexity of generated objects is limited to existing datasets.
- Only sketch and extrusion commands are modeled.
- Invalid rate and accuracy degrade somewhat on real-world images compared to synthetic.
Experimental setup
Datasets: DeepCAD RealCAD
Benchmarks: DeepCAD test set RealCAD
Metrics: Command Accuracy Parameter Accuracy Median Chamfer Distance Invalid Rate
Baselines: DeepCAD HNC-CAD Img2CAD One-2-3-45 Wonder3D TripoSR
Builds on: DeepCAD latent diffusion direct preference optimization DALL-E 2 diffusion transformer
Closest papers by meaning
- MiCADangelo: Fine-Grained Reconstruction of Constrained CAD Models from 3D Scans (NeurIPS 2025 · similarity 0.89)
- CADDreamer: CAD Object Generation from Single-view Images (CVPR 2025 · similarity 0.88)
- From 2D CAD Drawings to 3D Parametric Models: A Vision-Language Approach (AAAI 2025 · similarity 0.88)
- CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention (CVPR 2024 · similarity 0.88)
- cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning (ICLR 2026 · similarity 0.87)
- CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.