Research map / Large language models / Reasoning and chain of thought

Monitoring the Internal Monologue: Probe Trajectories Reveal Reasoning Dynamics

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Probes applied per token to LRM hidden states (prompt and CoT) yield cumulative max-pooled probe trajectories whose signal-processing features predict harmfulness and math-error outcomes, with max pooling far outperforming average pooling and last-token probing.

Problem

CoT is often unfaithful to the final output, so text-based CoT monitoring has a performance ceiling; the paper asks whether future LRM behavior (harmfulness, math correctness) can be forecast from hidden representations during prompt and CoT generation.

Approach

Track a multi-layer MIL meta-probe's prediction at every token using cumulative max-pooling to form probe trajectories, then extract signal-processing features (volatility, trend, steady-state) from these trajectories; also show template-based training data nearly matches exact model-response data and that max pooling is critical.

Key results

• Max-pooled template-based probes reach 94.44%-96.64% harmfulness AUROC on WildGuardTest across four reasoning models, while last-token and average pooling stay near 50%.
• Qwen3-14B reaches 95.91% AUROC with template data versus 97.14% with exact model messages.
• Trajectory features improve math-error prediction on GSM8K by up to ~17 percentage points over static max-pooled probes, with smaller gains on MATH.

Surprising findings

• Average-pooling and last-token probes collapse to near-random AUROC (around 50%) despite strong max-pooled performance.
• Template-based training data with no model generation achieves near-parity with exact model-response training data.

Limitations

• Evaluated only on models up to 14B parameters.
• Only two behavioral domains (harmfulness and math correctness) tested.
• Harmfulness labels rely on the WildGuard classifier, which may introduce label noise.

Experimental setup

Datasets: WildGuardMix Aegis ProcessBench GSM8K MATH

Benchmarks: WildGuardMix Aegis GSM8K MATH

Metrics: AUROC accuracy F1 balanced accuracy average precision

Baselines: LLM-as-a-Judge (Qwen3-8B) Last-Token probe Average Pooling probe 1D CNN trajectory extractor

Builds on: Representation Engineering WildGuard ProcessBench

Closest papers by meaning

• Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning (ICLR 2026 · similarity 0.89)
• Modeling Hierarchical Thinking in Large Reasoning Models (ICML 2026 · similarity 0.88)
• How Far Ahead Do LLMs Plan? Uncovering the Latent Horizon in Chain-of-Thought Reasoning (ICML 2026 · similarity 0.88)
• Probing the Trajectories of Reasoning Traces in Large Language Models (NeurIPS 2026 · similarity 0.88)
• Thought Branches: Interpreting LLM Reasoning Requires Resampling (ICLR 2026 · similarity 0.88)
• What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT (ICML 2026 · similarity 0.87)