Research map / Image generation / Text-to-image diffusion

Text-to-CAD Evaluation with CADTests

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: T2I evaluation and controllable adapters · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces CADTests, executable CadQuery-based property tests, and CADTestBench, a test-based Text-to-CAD benchmark built with an LLM planner, mutation analysis and iterative refinement, and shows test feedback improves CAD code generation.

Problem

Text-to-CAD evaluation relies on geometric similarity (Chamfer Distance, IoU) against a single reference model, which ignores valid design variations and cannot verify prompt-specified requirements explicitly.

Approach

Treat Text-to-CAD as program synthesis and evaluate generated B-reps with executable property-based tests (CADTests) written in CadQuery, whose test suites are synthesized by an LLM planner and refined via mutation analysis against passing and mutant sets.

Key results

• Claude-4.6-Sonnet with CADTest + Log baseline reaches 0.810 pass rate and 0.962 requirement score on abstract prompts, versus 0.715 PR for training-free Claude Skilled-style baseline.
• The refinement loop raises mutation score above 90% for both detailed and abstract prompts after 4 rounds.
• CADTests-based requirement score achieves AUC 0.928 against human labels, versus 0.663 for Chamfer Distance and 0.665 for CLIP Score.

Surprising findings

• Fine-tuned Text2CAD underperforms training-free VLM baselines on CADTestBench, and CADRille produced degenerate solutions and was excluded.
• Visual feedback via multi-view renderings does not improve performance over ReAct alone.

Limitations

• Test suites can be exploited, leading to misclassifications due to limited coverage; discriminative power is finite.
• CADTest verification may diverge from human judgement (human study kappa only 0.54).
• Mutants and tests are generated by LLM planners, so benchmark depends on planner quality and is built from 200 CADPrompt samples.

Experimental setup

Datasets: CADPrompt

Benchmarks: CADTestBench

Metrics: pass rate requirement score invalidity rate validity soundness mutation score cohen's kappa AUC accuracy

Baselines: Text2CAD K-shot Skilled ReAct ReAct + Vision CADCodeVerify CADTest CADTest + Log

Builds on: CADPrompt CadQuery ReAct Text2CAD CADCodeVerify

Closest papers by meaning

• Text2CAD: Generating Sequential CAD Designs from Beginner-to-Expert Level Text Prompts (NeurIPS 2024 · similarity 0.89)
• CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward (NeurIPS 2025 · similarity 0.87)
• Generating CAD Code with Vision-Language Models for 3D Designs (ICLR 2025 · similarity 0.87)
• CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing (ICML 2025 · similarity 0.87)
• Clarify Before You Draw: Proactive Agents for Robust Text-to-CAD Generation (ICML 2026 · similarity 0.86)
• GeoCAD: Local Geometry-Controllable CAD Generation with Large Language Models (NeurIPS 2025 · similarity 0.86)