Research map / Vision-language and multimodal / Multimodal LLMs

Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Shape-of-Thought, a visual chain-of-thought that fine-tunes Bagel-7B to interleave text plans and rendered 2D assembly states, trained on a 26K PartNet-derived trace dataset (SoT-26K) and evaluated on T2S-CompBench.

Problem

Text-to-image generators are brittle on compositional structural constraints such as generative numeracy, attribute binding, and part-level relations.

Approach

Treat rendered shape generation as progressive part assembly: a unified multimodal autoregressive model decides each assembly step in text and then grounds it by generating the rendered intermediate state, trained with process supervision from PartNet hierarchies rendered in Blender.

Key results

• Bagel-7B-SoT reaches 88.44 CN and 84.76 VT on T2S-CompBench, versus 64.26 CN and 65.42 VT for direct Bagel-7B generation.
• Visual intermediate states raise trace stability (TS) from 32.71 (text-only CoT) to 91.30 and rationale alignment (RA) to 79.19.
• Bagel-7B-SoT gets human score 4.08 vs 3.91 for Meshy 6 and 3.12 for direct Bagel-7B.

Surprising findings

• Removing interleaved visual thoughts drops the text-only CoT variant to the same numbers as the CoT baseline, and free-form rationales keep structural scores close while lowering RA sharply (55.47).

Limitations

• Single canonical front view for main training and evaluation; multi-view results show failure cases.
• Dense visual traces are very expensive to train (saturating 32 H100 GPUs).
• Learned procedure is not physically interpretable; no contact or support constraints enforced.

Experimental setup

Datasets: PartNet SoT-26K

Benchmarks: T2S-CompBench

Metrics: CN SF AF CP VT RA TS human score (1-5)

Baselines: Bagel-7B Bagel-7B-CoT Shap-E LGM L3GO Meshy 6

Builds on: Bagel PartNet T2I-CompBench++ Chain-of-Thought prompting

Closest papers by meaning

• Factuality Matters: When Image Generation and Editing Meet Structured Visuals (ICLR 2026 · similarity 0.88)
• Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.87)
• GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models (CVPR 2026 · similarity 0.86)
• When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought (CVPR 2026 · similarity 0.86)
• CADMorph: Geometry‑Driven Parametric CAD Editing via a Plan–Generate–Verify Loop (NeurIPS 2025 · similarity 0.86)
• SO-Bench: A Structural Output Evaluation of Multimodal LLM (CVPR 2026 · similarity 0.86)