Research map / Large language models / Agents, code and math
Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes introspective planning, which builds a retrieval knowledge base of LLM-generated post-hoc rationales conditioned on human-validated plans, and combines it with conformal prediction to reduce over-asking while keeping coverage guarantees for robot task planning.
Problem
LLM-based robot planners hallucinate or are overconfident, and natural-language ambiguity makes them either unsafe or over-ask; conformal prediction gives guarantees but is conservative and produces large prediction sets.
Approach
Construct a knowledge base of introspective reasoning examples by prompting the LLM to rationalize ground-truth valid/invalid plan labels, retrieve the most similar examples at inference to guide reasoning about compliance and safety, and calibrate conformal prediction on the rationale-augmented confidence scores to tighten bounds.
Key results
- On Safe Mobile Manipulation with GPT-4, Ours (Direct) reaches 93.0% exact set rate and 96.5% success rate versus 37.5% ESR for KnowNo (Conformal).
- Ours (Direct) on GPT-4 Mobile Manipulation achieves 94.5% exact set rate vs 84.0% for Retrieval-Q-CoT and 37.0% for KnowNo.
- With conformal prediction at 85% target, Ours (Conformal) on GPT-4 Safe Mobile Manipulation attains 87.5% success rate with 6.4% overask, lower than KnowNo's 51.3%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use introspective planning with direct prediction rather than Retrieval-Q-CoT | GPT-4 Turbo planner · Safe Mobile Manipulation | ESR: 81.5 -> 93.0 | Helped |
| Use introspective planning with direct prediction rather than Retrieval-Q-CoT | GPT-4 Turbo planner · Safe Mobile Manipulation | UCR: 4.5 -> 0.5 | Helped |
| Use introspective planning with direct prediction rather than KnowNo conformal prediction | GPT-4 Turbo planner · Safe Mobile Manipulation | NCR: 51.0 -> 5.5 | Helped |
| Use conformal prediction with introspective planning rather than KnowNo conformal prediction | GPT-4 Turbo planner · Safe Mobile Manipulation | ESR: 37.5 -> 58.0 | Helped |
| Use conformal prediction instead of direct prediction | Introspective planning · Safe Mobile Manipulation | SR: 96.5 -> 87.5 | Hurt |
| Increase the knowledge base size (ablation) | Introspective planning · Mobile Manipulation | SR | Mixed |
| Increase the knowledge base size (ablation) | Introspective planning · Safe Mobile Manipulation | SR | Mixed |
| Use multi-label rather than single-label conformal prediction (ablation) | Introspective conformal prediction · Ambiguous tasks | prediction set conservativeness | Hurt |
Surprising findings
- Retrieval-Q-CoT and Auto-CoT, which use retrieval augmentation, gave little improvement over simpler prompting, attributed to misleading self-generated knowledge without human grounding.
- Direct prediction with introspection outperforms conformal prediction on exact set rate, but direct prediction does not guarantee success, showing a trade-off.
Limitations
- Substantial performance gap between direct and conformal prediction.
- Single-label conformal prediction assumes mutually exclusive options; multi-label variant produced overly conservative sets.
- Method cannot differentiate between distinct types of uncertainty, a concern for safety-critical deployment.
Experimental setup
Datasets: Mobile Manipulation (KnowNo dataset, 400 calibration, 200 test) Tabletop Rearrangement (400 calibration, 200 test)
Benchmarks: Mobile Manipulation Safe Mobile Manipulation Tabletop Rearrangement
Metrics: success rate help rate exact set rate non-compliant contamination rate unsafe contamination rate overask rate overstep rate unsafe rate
Baselines: KnowNo Prompt Set Prompt Set + CoT Retrieval-Q-CoT Auto-CoT
Builds on: KnowNo Auto-CoT Retrieval-Q-CoT chain-of-thought conformal prediction
Closest papers by meaning
- Towards Reliable LLM-based Robots Planning via Combined Uncertainty Estimation (NeurIPS 2025 · similarity 0.92)
- Tru-POMDP: Task Planning Under Uncertainty via Tree of Hypotheses and Open-Ended POMDPs (NeurIPS 2025 · similarity 0.88)
- From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents (ICLR 2026 · similarity 0.87)
- Planning in the Dark: LLM-Symbolic Planning Pipeline Without Experts (AAAI 2025 · similarity 0.87)
- Non-myopic Generation of Language Models for Reasoning and Planning (ICLR 2025 · similarity 0.87)
- Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verification (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.