Research map / Large language models / Instruction and fine-tuning

CReFT-CAD: Boosting Orthographic Projection Reasoning for CAD via Reinforcement Fine-Tuning

NeurIPS 2025 · Large language models › Instruction and fine-tuning · approach: Calibration and knowledge in LLM tuning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CReFT-CAD, a curriculum GRPO reinforcement fine-tuning plus supervised post-tuning for VLM-based orthographic projection reasoning, and releases the TriView2CAD benchmark (200k synthetic, 3k real-world drawings).

Problem

Orthographic projection reasoning for CAD requires precise dimension extraction and parametric editability, but standard 3D reconstruction pipelines yield imprecise dimensions, and SFT of VLMs tends to memorize patterns and generalize poorly OOD.

Approach

A curriculum of three GRPO tasks (dichotomous choice, multiple choice, CoT parameterization) with difficulty-aware rewards, followed by multi-task SFT for instruction following, trained on a constraint-driven synthetic dataset of bridge-pier three-view drawings.

Key results

• CReFT-CAD reaches 83.24% on in-domain test set (+Answered Pair, no reasoning guidance) and 84.03% with reasoning guidance, versus 38.88% for best off-the-shelf VLM (Qwen2.5-VL).
• On 3,000 real-world samples, the baseline Qwen2.5-VL gets 13.47%, SFT gets 36.15%, and CReFT-based method gets 46.67%.
• Ablation with CoT: Task1+Task2+Task3 improves from 46.24% (w/o CoT) to 81.35% (with CoT).

Measured effects reported in the paper

Surprising findings

• Adding Task 2 (multiple-choice) on top of Task 3 without CoT drops accuracy to 30.71% from 46.16%, and with CoT Task 2+Task 3 gives 67.75%, below Task 3 alone at 74.15%.
• Reasoning guidance text yields only modest gains for the fine-tuned model, and at times reduces performance for some baselines (e.g., InternVL3 with Attribute Explanation drops from 17.91 to 17.05 for Test Img style).

Limitations

• Accuracy on complex real-world scenarios remains far from high, limiting OOD generalization.
• Evaluation is restricted to a single domain (prefabricated bridge piers) with 15-dimensional parameter space.
• Reward design and curriculum rely on engineered difficulty thresholds and warm-up sampling.

Experimental setup

Datasets: TriView2CAD

Benchmarks: TriView2CAD synthetic test set TriView2CAD real-world test set (3,000 samples)

Metrics: accuracy over 15 parameters

Baselines: Phi-3.5-Vision LLaVA-OneVision DeepSeek-VL InternVL2.5 InternVL3 Qwen2.5-Omni Qwen2.5-VL Gemini2.5 Pro Claude4 Vision GPT-4o Qwen2.5-VL (SFT)

Builds on: GRPO DeepSeek-R1-Zero GOT-OCR2.0 Qwen2.5-VL

Closest papers by meaning

• cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning (ICLR 2026 · similarity 0.87)
• From Intent to Execution: Multimodal Chain-of-Thought Reinforcement Learning for Precise CAD Code Generation (AAAI 2026 · similarity 0.86)
• ReCAD: Reinforcement Learning Enhanced Parametric CAD Model Generation with Vision-Language Models (AAAI 2026 · similarity 0.86)
• CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs (AAAI 2025 · similarity 0.86)
• CADMorph: Geometry‑Driven Parametric CAD Editing via a Plan–Generate–Verify Loop (NeurIPS 2025 · similarity 0.85)
• CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward (NeurIPS 2025 · similarity 0.84)