Research map / Multi-agent systems and games / LLM agents
Abstract Counterfactuals for Language Model Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Abstract Counterfactuals, which performs counterfactual inference over a high-level semantic abstraction of LM agent actions rather than over tokens, and mapping results back to actions, evaluated on MACHIAVELLI and text-generation tasks.
Problem
Token-level counterfactuals for language model agents are semantically inconsistent because LM actions lie in open-ended or choice-based spaces where token identity changes meaning across contexts.
Approach
Introduce an abstraction variable Y = f_Y(A, S, U_Y) capturing high-level action semantics; perform abduction on Y only (bypassing token-level Gumbel noise), derive counterfactual Y', then map back to actions via posterior P(A|Y', s'). Works with black-box LLM access and supervised or LLM-discovered abstractions.
Key results
- On gender steering with GPT2-XL, ACF reduces Abstraction Change Rate to 0.04 (supervised) and 0.12 (unsupervised) versus 0.40 and 0.38 for token-level counterfactuals.
- On GoEmotions with GPT2-XL, ACF achieves CPIR 0.96 versus 0.68 for TLCF under supervised abstraction.
- ACF achieves higher semantic tightness than TLCF in all settings (win-rate ST 0.72-0.82, p<0.001).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ACF instead of TLCF with supervised profession abstraction | GPT2-XL gender steering · Bios | ACR: 0.40 -> 0.04 | Helped |
| Use ACF instead of TLCF with unsupervised abstraction | GPT2-XL gender steering · Bios | ACR: 0.38 -> 0.12 | Helped |
| Use ACF instead of TLCF with supervised profession abstraction | GPT2-XL gender steering · Bios | CPIR: 0.59 -> 0.98 | Helped |
| Use ACF instead of TLCF with unsupervised abstraction | GPT2-XL gender steering · Bios | CPIR: 0.73 -> 0.98 | Helped |
| Use ACF instead of TLCF | GPT2-XL gender steering, supervised and unsupervised abstractions · Bios | ST win rate: 0.78; 0.81 | Helped |
| Use ACF instead of TLCF with supervised emotion abstraction | GPT2-XL token replacement · GoEmotions | ACR: 0.32 -> 0.02 | Helped |
| Use ACF instead of TLCF with supervised emotion abstraction | Llama-3.2-1B token replacement · GoEmotions | ACR: 0.37 -> 0.05 | Helped |
| Use ACF instead of TLCF with unsupervised abstraction | GPT2-XL token replacement · GoEmotions | ACR: 0.54 -> 0.27 | Helped |
| Use ACF instead of TLCF with unsupervised abstraction | Llama-3.2-1B token replacement · GoEmotions | ACR: 0.67 -> 0.41 | Helped |
| Use ACF instead of TLCF with supervised emotion abstraction | GPT2-XL token replacement · GoEmotions | CPIR: 0.68 -> 0.96 | Helped |
| Use ACF instead of TLCF with supervised emotion abstraction | Llama-3.2-1B token replacement · GoEmotions | CPIR: 0.67 -> 0.97 | Helped |
| Use ACF instead of TLCF with unsupervised abstraction | GPT2-XL token replacement · GoEmotions | CPIR: 0.48 -> 0.87 | Helped |
Limitations
- Requires multiple samples from the black-box LM, which is computationally costly.
- Defining a suitable abstraction distribution can be difficult in some settings.
- MACHIAVELLI evaluation is only illustrative due to deterministic environment, with no quantitative results there.
Experimental setup
Datasets: MACHIAVELLI BiosBias GoEmotions
Benchmarks: MACHIAVELLI BiosBias GoEmotions
Metrics: Abstraction Change Rate Counterfactual Probability Increase Rate Semantic Tightness
Baselines: token-level counterfactuals (TLCF) MiMiC
Builds on: structural causal models Gumbel-Max SCM token-level counterfactuals MiMiC
Closest papers by meaning
- Should I Have Expressed a Different Intent? Counterfactual Generation for LLM-Based Autonomous Control (ICML 2026 · similarity 0.91)
- Counterfactual reasoning: an analysis of in-context emergence (NeurIPS 2025 · similarity 0.88)
- Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations (ICML 2024 · similarity 0.87)
- RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code (ICLR 2025 · similarity 0.87)
- Towards Unifying Evaluation of Counterfactual Explanations: Leveraging Large Language Models for Human-Centric Assessments (AAAI 2025 · similarity 0.87)
- DRIFT-BENCH: Diagnosing CoopeRative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interaction (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.