Research map / Multi-agent systems and games / LLM agents

Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Reflective Test-Time Planning, where an embodied LLM agent combines best-of-N internal-reflection action selection with test-time training on retrospective and external verbal reflections, evaluated on a new long-horizon household benchmark and a MuJoCo cupboard fitting task.

Problem

Embodied LLM agents are static at deployment and repeat the same mistakes across trials; they cannot turn reflections on failures into persistent learning within an episode.

Approach

Unifies reflection-in-action (sampling N candidate actions and selecting the best by internal verbal reflection scores) with reflection-on-action (test-time training of the action policy via REINFORCE and the internal reflection model via supervised learning, using external and hindsight retrospective reflections as self-supervised signals).

Key results

• Reflective Test-Time Planning reaches 33.65% average success on Long-Horizon Household versus 9.92% for Reflexion and 11.20% for Self-Refine.
• On Cupboard Fitting the full method achieves 60.2% fit rate and 25.3% correct rate, versus 44.5% fit rate with both RIA and ROA removed.
• Retrospective reflection TTT reaches 33.7% average at 1x overhead versus 15.6% for TTT on per-step external reflection at roughly 4x overhead.

Surprising findings

• Removing RIA alone or ROA alone sometimes performs worse than removing both on the Household benchmark (e.g., Preparation 3.17% without RIA vs 11.1% without both).
• GPT-5 and Gemini 2.5 Pro zero-shot baselines score only about 20% average, and the 7B Qwen2.5-VL degrades relative to 3B after SFT.

Limitations

• Test-time overhead of about 3x per step for the full method.
• Reflection quality depends on GPT-5-generated SFT bootstrap data and an external reflection model; results mostly on self-designed benchmarks.
• Performance is modest in absolute terms (33.7% average on Household, 19.5% on HM3D).

Experimental setup

Datasets: BEHAVIOR-1K scenes HM3D GPT-5-generated Long-Horizon Household trajectories

Benchmarks: Long-Horizon Household (BEHAVIOR-1K based) MuJoCo Cupboard Fitting HM3D zero-shot Preparation tasks real-robot Franka Panda cupboard fitting

Metrics: success rate fit rate correct rate

Baselines: Reflexion Self-Refine ReflectVLM PPO DreamerV3 3DLLM-Mem GPT-5 Gemini 2.5 Pro Qwen2.5 LG-MCTS

Builds on: Reflexion LLaVA-3D BEHAVIOR-1K double-loop learning

Closest papers by meaning

• Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making (NeurIPS 2024 · similarity 0.87)
• ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making (NeurIPS 2026 · similarity 0.86)
• PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks (ICLR 2025 · similarity 0.86)
• RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation (NeurIPS 2025 · similarity 0.86)
• Reflective Prompted Policy Optimization: Trajectory-Grounded Revision and Salience Bias (NeurIPS 2026 · similarity 0.86)
• Towards Reliable Code-as-Policies: A Neuro-Symbolic Framework for Embodied Task Planning (NeurIPS 2025 · similarity 0.85)