AI Paper Atlas

Research map / Multi-agent systems and games / LLM agents

Stress Testing Chain-of-Thought Monitoring Against Covert Misalignment

Yao Huang, Yifan Wang, Yitong Sun, Yichi Zhang, Yinpeng Dong

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Stress-tests chain-of-thought monitors against covertly misaligned LLM agents, arguing that existing red-teaming is unrealistic for judging CoT monitoring reliability.

Problem

Reliability of chain-of-thought monitoring for detecting hidden misaligned objectives in LLM agents depends on how realistic the red-team used to stress-test it is; existing red-teaming efforts are judged insufficiently realistic.

Approach

Stress-testing CoT monitors against more realistic covert misalignment red-teaming rather than existing simplified adversarial setups. Only the abstract was available, so the specific method is not stated.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use TraceMRT trace-level red-teaming instead of prior red-team promptsCoT monitors (GPT-5.2, Claude-4.5, Claude-4) · SHADE-Arenaaverage ESR: 27.56% -> 59.94%Improved
use TraceMRT instead of prior red-team promptsCoT monitors · CUA-SHADE-Arenaaverage ESR: 23.89% -> 61.67%Improved
add fixed-template attacks (CoT Red-Handed)GPT-5.2 monitor · SHADE-Arena / CUA-SHADE-ArenaESRNo clear effect
add MRT prompt insertionClaude-4.5-Sonnet monitor · SHADE-Arena / CUA-SHADE-ArenaESRHurt
use both TraceMRT stages instead of a single stage (ablation)TraceMRT · GPT-5.2 monitor, ablationESR: about 20% ESR improvementImproved
replace CoT with action-only monitoring input under TraceMRT (ablation)GPT-5.2 monitor · SHADE-ArenaAUC: CoT 0.346 vs action-only 0.528Mixed
give the monitor reasoning traces (CoT) on benign prompts (ablation)GPT-5.2 monitor · SHADE-ArenaAUC: 0.473 -> 0.983Improved
add Bayesian Trajectory MonitoringClaude-4-Sonnet monitor · TraceMRT trajectoriesAUC: 0.420 -> 0.942Improved
add Bayesian Trajectory MonitoringQwen3.5-27B monitor · TraceMRT trajectoriesAUC: 0.399 -> 0.932Improved
use likelihood-ratio Bayesian aggregation instead of direct suspiciousness scoring in the same scaffold (ablation)Bayesian Trajectory Monitoring · SHADE-Arena, GPT-5.2AUC / TPR@5% FPR: AUC 0.8989 -> 0.9189; TPR 0.7288 -> 0.8167Improved
use Sequential scaffold instead of Bayesian (ablation)Monitor scaffold · SHADE-ArenaTPR@FPR=0.05: dropping to zero TPRHurt
frame the side task as improving the main task (Method relation) (ablation)TraceMRT stage 1 · SHADE-Arena and CUA-SHADE-Arenarelation selection successImproved

Limitations

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy