Research map / Vision-language and multimodal / Multimodal LLMs

Error Notebook-Guided, Training-Free Part Retrieval in 3D CAD Assemblies via Vision-Language Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a training-free two-stage VLM pipeline for specification-driven part retrieval in CAD assemblies, using an Error Notebook of corrected CoT trajectories filtered by a grammar-constraint verifier and retrieved via RAG as few-shot exemplars.

Problem

Retrieving the part identifiers in long, non-natural-language CAD assembly metadata that match a natural-language relational specification, where token limits are exceeded and proprietary VLMs cannot be fine-tuned.

Approach

Build an inference-time Error Notebook of reflection-corrected CoT trajectories (kept only if they pass a deterministic grammar-constraint check) and retrieve specification-similar entries as few-shot exemplars, combined with a first-stage VLM that converts part images into textual descriptions.

Key results

• GPT-4o (Omni) improves from 41.7% to 65.1% overall on the human preference dataset with Error Notebook (+23.4 points).
• The sGC grammar-constraint filter adds up to +4.5 points on the human preference dataset (e.g., 65.1% to 66.8% for GPT-4o).
• Qwen2-VL-2B with a GPT-4o-built Error Notebook (gE-Notebook+sGC) reaches 14.6% on the human preference dataset, up from 1.5% without the notebook.

Surprising findings

• Number of retrieved exemplars has only a minor effect on accuracy once an Error Notebook is used.
• Error Notebook inference did not become slower despite roughly doubling prompt tokens (6.50s vs 8.04s without).

Limitations

• Accuracy remains low in absolute terms for large assemblies (e.g., 5-16% for more than 50 parts with GPT-4o).
• Open-source 2B model often omits the explicit Final Answer marker, so strict grammar filtering discards many valid traces.
• Many VLM calls are needed per assembly (one per part for descriptions), making the first stage costly for large assemblies.

Experimental setup

Datasets: Fusion 360 Gallery Assembly Dataset (a1.0.0_00, 752 assemblies)

Benchmarks: self-generated dataset human preference dataset

Metrics: accuracy recall F1

Baselines: standard few-shot self-consistency w/o E-Notebook (direct CoT) image-only reasoning Gemini 1.5 Pro Gemini 2.0 Flash Gemini 2.5 Pro GPT-4o mini

Builds on: chain-of-thought retrieval-augmented generation self-consistency Fusion 360 Gallery Dataset

Closest papers by meaning

• CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention (CVPR 2024 · similarity 0.87)
• Imagine: Image-Guided 3D Part Assembly with Structure Knowledge Graph (AAAI 2025 · similarity 0.86)
• CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation (CVPR 2025 · similarity 0.86)
• cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning (ICLR 2026 · similarity 0.85)
• CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs (AAAI 2025 · similarity 0.85)
• CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images (CVPR 2025 · similarity 0.85)