Research map / Large language models / Language, safety and interpretability
Cognitive Fatigue in Autoregressive Transformers: Formalization and Measurement
Riju Marwah, Ritvik Garimella, Vishal Pallagani, Atishay Jain, Michael Charles Stewart, Amit Sheth
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| aggregate attention, drift and entropy rather than using entropy alone (ablation) | Fatigue Index · HotpotQA | AUROC: 0.954 -> 0.976 | Improved |
| aggregate signals rather than using drift alone (ablation) | Fatigue Index · HotpotQA | AUROC: 0.929 -> 0.976 | Improved |
| aggregate signals rather than inverted attention alone (ablation) | Fatigue Index · HotpotQA | AUROC: 0.307 -> 0.976 | Improved |
| add hysteresis thresholds (0.50/0.40) to the alert rule | FI alerting · HotpotQA | alert flips per generation: 18.934 -> 1.567 | Improved |
| add hysteresis thresholds to the alert rule | FI alerting · TriviaQA | alert flips per generation: 20.889 -> 1.433 | Improved |
| add hysteresis thresholds to the alert rule | FI alerting · SQuAD | alert flips per generation: 21.085 -> 1.674 | Improved |
| compute FI over full generation instead of first 20 tokens (ablation) | FI vs repetition · HotpotQA | Spearman correlation: 0.425 -> 0.848 | Improved |
| compute FI over full generation instead of first 20 tokens (ablation) | FI vs repetition · SQuAD | Spearman correlation: 0.375 -> 0.856 | Improved |
| switch prompt style from zero-shot to short chain-of-thought | FI · seed 42 sequences | mean FI: 0.854 -> 0.858 | No clear effect |
| vary seed from 42 to 123 | FI · zero-shot prompts | mean FI: 0.854 -> 0.854 | No clear effect |
| increase context length from 192 to 1446 filler tokens | FI signals · context-length stress | prompt attention: len=192 vs len=1446 | Hurt |
| place evidence at front instead of middle or end (ablation) | attention to evidence · positional sensitivity | mean attention: 5--10x higher | Improved |
Closest papers by meaning
- Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking (ICLR 2026 · similarity 0.84)
- ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Model (CVPR 2025 · similarity 0.83)
- Beyond Accuracy: A Cognitive Load Framework for Mapping the Capability Boundaries of Tool-use Agents (AAAI 2026 · similarity 0.83)
- On the Entropy Calibration of Language Models (NeurIPS 2025 · similarity 0.83)
- Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale (NeurIPS 2025 · similarity 0.83)
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs (ICLR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
