Research map / Large language models / Language, safety and interpretability

How Much is Left? LLMs Linearly Encode Their Remaining Output Length

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Pruning and circuits in LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains minimal linear probes on frozen hidden states of three 7-8B instruction models to decode remaining output length, finding prompt-end decodability, asymmetric cross-dataset transfer, and qualitative upward shifts at retraction tokens.

Problem

Whether LLMs maintain an internal estimate of how many tokens of their response remain, rather than length being a downstream statistical regularity of decoding.

Approach

Linear probes regress remaining token count r_t = T - t from frozen residual-stream states at every position, compared against a constant-median baseline and an exact-countdown predictor seeded by a prompt-end length probe; cross-dataset matrices test transfer, and per-position updates are inspected at retraction tokens.

Key results

• Completion Length Probe on Llama-3.1-8B reduces prompt-end MAE on Countdown to 5.27 tokens versus 150.18 for the constant-median baseline.
• Remaining Count Probe trained on natural-language data beats the constant baseline on most natural-language and synthetic targets; Llama GSM8K per-token MAE 36.59 vs 70.15 baseline.
• Synthetic-trained probes (Count, Countdown) fail to transfer to natural-language targets, e.g. Count-trained probe scores 228.93 on MMLU-Pro vs 176.54 baseline on Llama.

Surprising findings

• Probes trained on natural-language datasets transfer back to synthetic Count/Countdown targets while synthetic-trained probes do not transfer to natural language.
• Layer-0 (embedding) probes sit near the constant baseline, suggesting the signal is a processed representation rather than current-token identity.

Limitations

• Decodability does not establish causal use; no activation patching was performed.
• Dynamic re-estimation evidence is a qualitative curated gallery of high-loss examples without a length-matched control.
• Only 7-8B instruction-tuned models; Mistral/TriviaQA omitted and cross-dataset matrices cover five datasets for Llama and Mistral.

Experimental setup

Datasets: Count Countdown GSM8K MATH MMLU-Pro OpenThoughts-1k TriviaQA

Benchmarks: Count Countdown GSM8K MATH MMLU-Pro OpenThoughts-1k TriviaQA

Metrics: per-token MAE prompt-end absolute error Cohen's kappa

Baselines: constant-median statistical baseline exact countdown predictor

Builds on: linear probing linear representation hypothesis s1 budget forcing

Closest papers by meaning

• LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs (ICLR 2025 · similarity 0.88)
• LIFEBENCH: Evaluating Length Instruction Following in Large Language Models (NeurIPS 2025 · similarity 0.88)
• The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs (ICLR 2026 · similarity 0.87)
• The Role of Sparsity for Length Generalization in LLMs (ICML 2025 · similarity 0.87)
• When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · similarity 0.87)
• Do Language Models Use Their Depth Efficiently? (NeurIPS 2025 · similarity 0.87)