Research map / Vision-language and multimodal / Robot action and navigation
Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Articulate-Anything uses a Gemini Flash-1.5 vision-language actor-critic system that generates Python code compiled to URDF, retrieving meshes from PartNet-Mobility to articulate objects from text, image, or video inputs.
Problem
Creating articulated 3D simulation assets requires heavy manual annotation; prior automatic articulation methods handle few object categories and only simple inputs like bounding boxes or static images.
Approach
Treat articulation as program synthesis: VLM actors write high-level Python code (compiled to URDF) for mesh retrieval, link placement and joint prediction, while VLM critics compare renders against input (image/video) and feed back corrections in a closed loop.
Key results
- On PartNet-Mobility, Articulate-Anything raises joint prediction success rate from 8.7-12.2% (prior work) to 75%.
- Iterative critic refinement gains about 5.8% for link placement and 2.4% for joint prediction accuracy.
- Robot policies trained in simulation on generated assets from in-the-wild videos succeed on a real Franka arm, matching human-annotated assets (100% success), saving about 40.58 minutes of human labor per task.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Articulate-Anything instead of prior work | PartNet-Mobility articulation · PartNet-Mobility | joint prediction success rate: 8.7--12.2% -> 75% | Helped |
| Provide richer grounded input modalities (text, image, or video rather than text alone) (ablation) | Articulate-Anything · Faucet (link placement) and StorageFurniture (joint prediction) | success rate | Helped |
| Iteratively refine predictions using critic feedback (ablation) | Articulate-Anything actor-critic system · PartNet-Mobility articulation | accuracy: +5.8% link placement; +2.4% joint prediction | Helped |
| Increase the number of in-context prompting examples (ablation) | Articulate-Anything actor-only system · StorageFurniture | success rate | Helped |
| Change the base vision-language model (ablation) | Articulate-Anything actor-only system · StorageFurniture | success rate | No clear effect |
| Use mesh generation rather than mesh retrieval (ablation) | Articulate-Anything · In-the-wild objects | Chamfer distance: 0.1102 -> 0.0703 | Helped |
Surprising findings
- The VLM critic tends to overestimate success, with false positives being the largest disagreement with ground truth.
- Baselines produce invalid outputs at high rates (59.1% for URDFormer, 17.5% for Real2Code).
Limitations
- Relies on mesh retrieval from existing datasets, so assets are limited to library shapes rather than customized meshes.
- Small object parts and exotic irregular movements are harder to articulate.
- VLM critic overestimates success, missing subtle errors.
Experimental setup
Datasets: PartNet-Mobility
Benchmarks: PartNet-Mobility
Metrics: link placement success rate joint prediction success rate joint type error joint axis error joint origin error Chamfer distance
Baselines: URDFormer Real2Code
Builds on: URDFormer Real2Code
Closest papers by meaning
- IAAO: Interactive Affordance Learning for Articulated Objects in 3D Environments (CVPR 2025 · similarity 0.89)
- URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Model (NeurIPS 2025 · similarity 0.88)
- MeshArt: Generating Articulated Meshes with Structure-Guided Transformers (CVPR 2025 · similarity 0.87)
- AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models (NeurIPS 2025 · similarity 0.87)
- MagicArticulate: Make Your 3D Models Articulation-Ready (CVPR 2025 · similarity 0.87)
- InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.