Research map / Large language models / Language, safety and interpretability
Representational Curvature Modulates Behavioral Uncertainty in Large Language Models
Jack G. King, Evelina Fedorenko, Eghbal A. Hosseini
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add trajectory-subspace perturbation instead of full-space, random-subspace or activation-subspace perturbation (ablation) | Curvature-entropy perturbation analysis · LAMBADA, middle layers (GPT-2 XL, Pythia-2.8B) | correlation between delta curvature and delta entropy | Improved |
| restrict perturbation to the planar subspace of the two most recent displacements (ablation) | Curvature-entropy perturbation analysis · LAMBADA, middle layers | correlation between delta curvature and delta entropy | Improved |
| add curvature-penalty regularization (untangling) at layers 7 and 8 during training | GPT-2 Small trained from scratch · token-level entropy on three validation sets | token-level entropy | Improved |
| add reversed curvature penalty (tangling) during training | GPT-2 Small trained from scratch · token-level entropy on three validation sets | token-level entropy | Mixed |
| add curvature regularization (untangling or tangling) | GPT-2 Small trained from scratch · validation set | validation loss | No clear effect |
| use window size larger than 3 for contextual curvature (ablation) | Contextual curvature regression · LAMBADA, GPT-2 XL and Pythia-2.8B | Pearson r predicting entropy | Hurt |
| apply trajectory-aligned perturbations at late layers instead of middle layers (ablation) | Pythia-2.8B perturbation analysis · LAMBADA, late layers | correlation between delta curvature and delta entropy | Hurt |
| use contextual magnitude or distance instead of curvature as predictor (ablation) | Entropy regression · LAMBADA and short-context UD sentences | Pearson r predicting entropy | Hurt |
Closest papers by meaning
- Tracing the Representation Geometry of Language Models from Pretraining to Post-training (NeurIPS 2025 · similarity 0.90)
- Less is More: Local Intrinsic Dimensions of Contextual Language Models (NeurIPS 2025 · similarity 0.89)
- The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations (NeurIPS 2026 · similarity 0.86)
- Remarkable Robustness of LLMs: Stages of Inference? (NeurIPS 2025 · similarity 0.86)
- Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale (NeurIPS 2025 · similarity 0.86)
- Integrating Local and Global Entropy for Uncertainty Quantification in LLMs (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
