Research map / Multi-agent systems and games / LLM agents

Should I Have Expressed a Different Intent? Counterfactual Generation for LLM-Based Autonomous Control

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes conformal counterfactual generation (CCG), which models an LLM agent plus environment as an SCM, uses abduction via neural posterior estimation and test-time scaling, and calibrates acceptance/stopping rules to guarantee counterfactual report sets contain an admissible report, evaluated on a 5G ns-3 network control task.

Problem

Users of LLM-based agents controlling environments want to know what would have happened under a different intent, but re-running the closed loop is either unfaithful or costly, and there are no reliability guarantees on counterfactual outputs.

Approach

Combine token-level Gumbel-Max counterfactual reasoning for the LLM with simulator-based abduction of environment noise (NPE), and apply conformal language modeling-style calibration over sampled counterfactual reports so the returned set contains an admissible counterfactual with high probability.

Key results

• CG achieves throughput MAE 0.15 vs 0.28 (IG) and 0.33 (SIG) and throughput cross-correlation peak 0.93 vs 0.78 and 0.70.
• In LLM-judge comparison over 100 cases, CG is selected as best match in 92% of cases, IG in 8%, SIG never.
• CCG keeps set loss close to the target epsilon and yields fewer excess samples than fixed-budget k-CG at matched set sizes.

Surprising findings

• Simulator fidelity improvements saturate, and the highest fidelity level Q=4 shows some degradation attributed to independent estimation of the exogenous noise.

Limitations

• Evaluated only on a single network control use case with a simplified digital twin.
• Admission function relies on an LLM judge (GPT-4) rather than human-verified correctness.
• Guarantees depend on calibration data representative of test distribution and only cover the defined admission function.

Experimental setup

Datasets: 300 factual-counterfactual prompt pairs generated with GPT-4 and simulated with ns-3

Benchmarks: 5G gNB ns-3 counterfactual KPI and report evaluation on 300 prompt pairs (150 test, 150 calibration)

Metrics: MAE cross-correlation peak crossing-level error set loss relative excess number of samples set size fraction preferred by judge

Baselines: Interventional Generation (IG) Simulated Interventional Generation (SIG) k-CG fixed-budget sampling

Builds on: conformal language modeling Gumbel-max counterfactual generation for LLMs neural posterior estimation Bonferroni correction fixed-sequence testing

Closest papers by meaning

• Abstract Counterfactuals for Language Model Agents (NeurIPS 2025 · similarity 0.91)
• Constitutional Black-Box Monitoring for Scheming in LLM Agents (ICML 2026 · similarity 0.86)
• DRIFT-BENCH: Diagnosing CoopeRative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interaction (ICML 2026 · similarity 0.86)
• From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems (ICML 2024 · similarity 0.85)
• Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity (NeurIPS 2024 · similarity 0.85)
• From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents (ICLR 2026 · similarity 0.85)