Research map / Vision-language and multimodal / Robot action and navigation
Describe-Then-Act: Proactive Agent Steering via Distilled Language-Action World Models
Massimiliano Pappa, Luca Romani, Valentino Sacco, Alessio Palma, Stéphane Lathuilière, Fabio Galasso, Xavier Alameda-Pineda, Indro Spinelli
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the three-stage curriculum (projector alignment, description distillation) before verdict training (ablation) | DILLO-1B · LIBERO avg | success rate (%): 71.6 -> 78.0 | Improved |
| add the three-stage curriculum (ablation) | DILLO-4B · LIBERO avg | success rate (%): 73.5 -> 78.9 | Improved |
| add the three-stage curriculum (ablation) | DILLO-4B · LIBERO-Spatial | success rate (%): 54.0 -> 73.0 | Improved |
| add description generation objective to verdict training (ablation) | DILLO-4B · LIBERO avg | success rate (%): 72.2 -> 78.9 | Improved |
| add description generation objective to verdict training (ablation) | DILLO-1B · LIBERO avg | success rate (%): 75.1 -> 78.0 | Improved |
| use language-model DILLO-4B instead of latent MLP critic (ablation) | DILLO-4B · LIBERO avg | success rate (%): 74.2 -> 78.9 | Improved |
| steer base policy with latent MLP critic (ablation) | Base policy · LIBERO avg | success rate (%): 74.7 -> 74.2 | No clear effect |
| steer with DILLO-4B rejection sampling (N=5) | SAC policy · MetaWorld Soccer | success rate (%): 65.0 -> 80.0 | Improved |
| steer with DILLO-1B rejection sampling | ACT policy · LIBERO-90 | success rate (%): 71.4 -> 68.6 | Hurt |
| use heavier rejection gating threshold on LIBERO-90 (ablation) | DILLO · LIBERO-90 | success rate: -4.4% | Hurt |
| steer LIBERO-10 policy with LIBERO-90-trained DILLO (cross-suite) | ACT policy · LIBERO-10 | success rate (%): 64.0 -> 69.5 | Improved |
| scale student from 1B to 4B (ablation) | DILLO · MetaWorld steering | success rate (%): Drawer-Open 70.0 -> 90.0; Sweep-Into 90.0 -> 85.0 | Mixed |
Closest papers by meaning
- ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response (ICML 2026 · similarity 0.88)
- RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.88)
- World–Value–Action Model: Implicit Planning for Vision–Language–Action Systems (NeurIPS 2026 · similarity 0.87)
- PROACT-Agent: Progressive Runtime Oversight and Active Circuit-breaking for Real-Time Safety (NeurIPS 2026 · similarity 0.87)
- Failing Forward: Adaptive Failure-Informed Learning for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.87)
- Latent Action Reparameterization for Efficient Agent Inference (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
