Research map / Vision-language and multimodal / Multimodal LLMs

cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes cadrille, a VLM (Qwen2-VL-2B) that maps point clouds, multi-view images, or text to executable CadQuery Python code, trained with SFT on procedurally generated data and then RL fine-tuning (Dr. CPPO/DPO) on unannotated handcrafted meshes.

Problem

Single-modality CAD reconstruction methods generalize poorly, and existing multimodal CAD reconstruction approaches lag far behind single-modal state of the art; SFT on procedural data also suffers from invalid outputs and domain gaps to real scans.

Approach

Use a single VLM to handle three input modalities by generating CadQuery Python code, train with SFT on large procedural data, then apply online RL (Dr. CPPO, a Dr. GRPO plus CPPO hybrid) with IoU-based rewards on handcrafted meshes that need no CAD sequence annotations.

Key results

• With RL (Dr. CPPO) on multi-view images, cadrille reaches 92.2 IoU and 0.0 IR on DeepCAD, versus 81.7... (SFT image baseline 86.1 IoU, 1.5 IR)
• On real-world CC3D point clouds, Dr. CPPO improves IoU from 61.8 (SFT) to 67.9 with IR 0.2%, versus CAD-Recode's 60.5 IoU.
• SFT alone on point clouds matches CAD-Recode (0.18 CD, 87.1 IoU on DeepCAD) while extending to images and text in one model.

Surprising findings

• RL fine-tuning on images alone improved point-cloud reconstruction, reaching state of the art on point clouds too.
• Naively mixing DeepCAD and CAD-Recode in SFT degraded results relative to CAD-Recode-only SFT.

Limitations

• Multimodal results are reported in zero-shot or single-model settings, and RL is applied only to image inputs, not point clouds or text.
• Failure cases remain for objects with complex, granular surfaces.
• Text-based reconstruction remains the hardest modality, with Long textual descriptions not fully specifying geometry.

Experimental setup

Datasets: DeepCAD CAD-Recode Fusion360 CC3D Omni-CAD

Benchmarks: DeepCAD Fusion360 CC3D Omni-CAD

Metrics: median Chamfer Distance IoU invalidity ratio mean Chamfer Distance inference time

Baselines: CAD-Recode CAD-SIGNet CADCrafter Text2CAD CAD-Coder Text-to-CadQuery CAD-MLLM LRM→CAD-Recode CADFusion CAD-GPT

Builds on: CAD-Recode Dr. GRPO CPPO DPO Qwen2-VL Text2CAD

Closest papers by meaning

• From Intent to Execution: Multimodal Chain-of-Thought Reinforcement Learning for Precise CAD Code Generation (AAAI 2026 · similarity 0.91)
• ReCAD: Reinforcement Learning Enhanced Parametric CAD Model Generation with Vision-Language Models (AAAI 2026 · similarity 0.90)
• MiCADangelo: Fine-Grained Reconstruction of Constrained CAD Models from 3D Scans (NeurIPS 2025 · similarity 0.89)
• CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention (CVPR 2024 · similarity 0.88)
• CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward (NeurIPS 2025 · similarity 0.88)
• Draw Step by Step: Reconstructing CAD Construction Sequences from Point Clouds via Multimodal Diffusion. (CVPR 2024 · similarity 0.88)