Research map / Large language models / Agents, code and math
Towards Reliable LLM-based Robots Planning via Combined Uncertainty Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CURE, which decomposes LLM planning uncertainty into epistemic (task clarity and task familiarity via RND) and intrinsic (expected success rate) components, and evaluates it on kitchen manipulation and tabletop rearrangement.
Problem
LLM-based robot planners produce overconfident plans; existing uncertainty estimates do not separate epistemic from intrinsic uncertainty, so their alignment with execution success is weak.
Approach
Decompose planning uncertainty into epistemic (task clarity from ambiguity detection plus task familiarity from random network distillation) and intrinsic (expected success rate from MLP heads on LLM hidden features) components, combined in a single formula.
Key results
- In the kitchen task, CURE-Ambiguity reaches Spearman 0.466 and SR-HR-AUC 0.547, versus 0.336 and 0.395 for the best baseline KnowNo.
- In tabletop rearrangement, CURE reaches Spearman 0.635 and SR-HR-AUC 0.732, versus 0.328 and 0.351 for the best baselines (Vanilla and KnowNo respectively).
- IntroPlan shows near-zero correlation (0.030, p=0.60) in the kitchen task.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CURE instead of KnowNo | LLM-based uncertainty estimation · Kitchen mobile manipulator | Spearman: 0.336 -> 0.454 | Helped |
| Use CURE instead of KnowNo | LLM-based uncertainty estimation · Kitchen mobile manipulator | SR-HR-AUC: 0.395 -> 0.534 | Helped |
| Add RND task-similarity component (ablation) | CURE · Kitchen mobile manipulator | Spearman: 0.417 -> 0.454 | Helped |
| Add RND task-similarity component (ablation) | CURE · Kitchen mobile manipulator | SR-HR-AUC: 0.483 -> 0.534 | Helped |
| Add LLM-based ambiguity assessment to CURE (ablation) | CURE-Ambiguity · Kitchen mobile manipulator | Spearman: 0.454 -> 0.466 | Helped |
| Add LLM-based ambiguity assessment to CURE (ablation) | CURE-Ambiguity · Kitchen mobile manipulator | SR-HR-AUC: 0.534 -> 0.547 | Helped |
| Use CURE instead of KnowNo | LLM-based uncertainty estimation · Tabletop rearrangement | Spearman: 0.302 -> 0.635 | Helped |
| Use CURE instead of KnowNo | LLM-based uncertainty estimation · Tabletop rearrangement | SR-HR-AUC: 0.351 -> 0.732 | Helped |
| Add RND task-similarity component (ablation) | CURE · Tabletop rearrangement | Spearman: 0.610 -> 0.635 | Helped |
| Add RND task-similarity component (ablation) | CURE · Tabletop rearrangement | SR-HR-AUC: 0.703 -> 0.732 | Helped |
Surprising findings
- The ambiguity-only method (0.433 Spearman) already outperforms all baseline uncertainty methods in the kitchen task, and CURE w/o sim is strong on its own.
- IntroPlan shows essentially no correlation with success in the kitchen task, which the authors attribute to its high initial success rate.
Limitations
- The predictive model must be pretrained on existing datasets and lacks generalizability to new task sets.
- Uncertainty estimates are not calibrated against actual success rates, requiring an extra calibration step.
- Hyperparameters alpha_2 and alpha_3 were tuned on the kitchen setting, and the framework yields high uncertainty in unseen environments.
Experimental setup
Datasets: KnowNo kitchen task specifications KnowNo tabletop rearrangement task definitions (PyBullet)
Benchmarks: Mobile manipulator kitchen task (KnowNo setup) PyBullet tabletop rearrangement (KnowNo setup)
Metrics: Spearman correlation p-value SR-HR-AUC
Baselines: KnowNo IntroPlan Multi-step Top-k CoT Vanilla Self-probing Self-probing-log
Builds on: Random Network Distillation KnowNo IntroPlan
Closest papers by meaning
- Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity (NeurIPS 2024 · similarity 0.92)
- From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents (ICLR 2026 · similarity 0.89)
- Tru-POMDP: Task Planning Under Uncertainty via Tree of Hypotheses and Open-Ended POMDPs (NeurIPS 2025 · similarity 0.88)
- PlanU: Large Language Model Reasoning through Planning under Uncertainty (NeurIPS 2025 · similarity 0.88)
- Do LLMs estimate uncertainty well in instruction-following? (ICLR 2025 · similarity 0.88)
- Position: Uncertainty Quantification Needs Reassessment for Large Language Model Agents (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.