Research map / Multi-agent systems and games / LLM agents

PRISM: Festina Lente Proactivity—Risk-Sensitive, Uncertainty-Aware Deliberation for Proactive Agents

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PRISM, a decision-theoretic gate with a fast/slow dual-process reasoner and RDC-filtered distillation into Qwen3-8B for proactive-agent intervention, reporting higher F1 and lower false alarms than baselines on ProactiveBench.

Problem

Proactive agents must decide whether and when to intervene under asymmetric costs of false alarms and missed help, while avoiding over-intervention and indiscriminate expensive reasoning.

Approach

A Bayes-optimal acceptance threshold tau(p_need) derived from false-alarm and miss costs gates interventions, a slow reasoning pass is invoked only near the decision boundary, and a student is distilled from a teacher using RDC-based curation of calibrated need/accept probabilities.

Key results

• PRISM (Qwen3-8B-PRISM) reaches F1 86.61% and false-alarm 22.94% on ProactiveBench held-out test, versus 66.47% F1 and 50.22% false alarm for Qwen2-7B-Proactive and 83.28% F1 for DeepSeek-R1.
• Slow-on-margin gating (delta=0.1) routes about 11% of cases to slow mode, raising F1 from 83.09% (fast-only) to 88.15% with about 20ms P95 latency overhead.
• Human expert evaluation with 229 events shows Cohen's kappa 0.71 between the LLM ensemble judge and human consensus; PRISM F1 84.85% vs DeepSeek-R1 82.05% under human labels.

Surprising findings

• The distilled 8B student surpasses its DeepSeek-R1 teacher on precision and false-alarm rate.
• Second-stage DFT after RDC-SFT lowered recall to 80.14% without precision gains.

Limitations

• Relies on LLM-as-judge labels, which may diverge from real user acceptance; human validation uses only 229 events.
• Assumes offline-calibrated probabilities remain stable under distribution shift and stationary costs.
• Distillation may inherit teacher biases; no hard safety constraints are integrated.

Experimental setup

Datasets: ProactiveBench

Benchmarks: ProactiveBench

Metrics: recall precision accuracy false-alarm rate F1 AUDBC ECE Brier score

Baselines: GPT-4o Claude-3.5-Sonnet GPT-4o-mini DeepSeek-R1 LLaMA-3.1-8B-Proactive Qwen2-7B-Proactive Qwen3-8B SFT DFT Weighted-SFT

Builds on: ProactiveBench Proactive Agent chain-of-thought Dynamic Fine-Tuning Reward-Weighted Regression

Closest papers by meaning

• ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response (ICML 2026 · similarity 0.85)
• ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal Evidence (NeurIPS 2026 · similarity 0.85)
• ExPO: Unlocking Hard Reasoning with Self-Explanation-Guided Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
• Tina: Tiny Reasoning Models via LoRA (ICLR 2026 · similarity 0.85)
• Selective Critique for Cost-Aware LLM Agents in Long-Horizon Decision Making (NeurIPS 2026 · similarity 0.85)
• ProRe: A Proactive Reward System for GUI Agents via Reasoner–Actor Collaboration (ICLR 2026 · similarity 0.85)