Research map / Large language models / Language, safety and interpretability
Exposing Attention Glitches with Flip-Flop Language Modeling
Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a 1-layer recurrent LSTM instead of a Transformer | FFLM language modeling · FFL(0.98) and FFL(0.1) out-of-distribution | read accuracy: 100% accuracy, on 100 out of 100 runs | Improved |
| Train on a uniform mixture including rare sparse and dense sequences | Transformer FFLM · FFL(0.9), FFL(0.98), and FFL(0.1) | read errors: In 6 out of 25 runs, we did not detect a single error. | Improved |
| Increase data, training steps, or network size | Transformer FFLM · Sparse and dense out-of-distribution sequences | read errors: orders of magnitude smaller | Mixed |
| Add attention-sharpening regularization | Transformer FFLM · Sparse out-of-distribution sequences | read errors: by an order of magnitude | Improved |
| Use embedding dropout 0.5 | Transformer FFLM · Sparse out-of-distribution sequences | read errors: 0.5 | Improved |
| Use non-trainable position embeddings | Transformer FFLM · Dense out-of-distribution sequences | read errors | Improved |
| Sharpen attention weights | Transformer FFLM · FFLM extrapolation | read errors | Mixed |
| Prompt with in-context demonstrations of flip-flop sequences | 10B-scale natural language models · Natural-language embeddings of flip-flop tasks | read accuracy: 10B | Mixed |
Closest papers by meaning
- Interpreting the Repeated Token Phenomenon in Large Language Models (ICML 2025 · similarity 0.88)
- Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration (ICML 2024 · similarity 0.88)
- An Analysis for Reasoning Bias of Language Models with Small Initialization (ICML 2025 · similarity 0.87)
- Attention Illuminates LLM Reasoning: The Uncovered Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization (ICML 2026 · similarity 0.87)
- Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models (ICLR 2024 · similarity 0.87)
- Reasoning with Latent Thoughts: On the Power of Looped Transformers (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
