Research map / Large language models / Retrieval and knowledge
Looking Inward: Language Models Can Learn About Themselves by Introspection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that LLMs finetuned to predict properties of their own hypothetical behavior outperform other models trained on the same data at self-prediction, providing evidence of introspection on simple tasks but not complex ones.
Problem
Whether LLMs can acquire knowledge about their own behavior that is not contained in or derivable from their training data (introspection), tested via self-prediction of hypothetical behavior.
Approach
Finetune a model to predict properties of its own hypothetical behavior and compare against a stronger model cross-trained on the same model's behavior; a self-advantage in prediction is taken as evidence of introspection, supplemented by calibration and behavioral-change tests.
Key results
- Llama 70B predicts its own behavior at 48.5% vs 31.8% for cross-trained GPT-4o on held-out tasks; GPT-4o self-predicts at 49.4% vs 36.6% for cross-trained Llama 70B.
- Self-prediction training improves calibration for Llama 70B, reducing MAD from 23.0 to 8.7, while cross-prediction reduces it only to 21.3.
- After behavioral change finetuning, GPT-4o predicts its new behavior at 35.4% vs 21.7% for its old behavior.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Self-prediction training rather than cross-prediction training | Llama 70B predicting its own behavior · Held-out tasks | accuracy: 31.8% -> 48.5% | Helped |
| Self-prediction training rather than cross-prediction training | GPT-4o predicting its own behavior · Held-out tasks | accuracy: 36.6% -> 49.4% | Helped |
| Self-prediction training | GPT-4o · Held-out datasets | accuracy: 32.6% -> 49.4% | Helped |
| Self-prediction training rather than cross-prediction training | GPT-3.5 predicting its own behavior · Held-out tasks | accuracy: +0.8% | Helped |
| Self-prediction training instead of no training | Llama 70B · Animal Sequence, second character | MAD: 23.0 -> 8.7 | Helped |
| Self-prediction training rather than cross-prediction training | Llama 70B · Animal Sequence, second character | MAD: 21.3 -> 8.7 | Helped |
| Fine-tune the self-prediction-trained model on Claude-3.5-Sonnet object-level behavior | GPT-4o · Behavioral Change, held-out prompts | accuracy predicting behavior: 21.7% -> 35.4% | Helped |
| Self-prediction training rather than cross-prediction training | Llama 70B predicting susceptibility to an 'Are you sure?' follow-up · Are you sure? bias detection | accuracy: 74.5% vs 76.5% | Hurt |
| Self-prediction fine-tuning instead of baseline fine-tuning | GPT-4o · Situational Awareness Dataset, overall plain | score: 0.49 -> 0.48 | No clear effect |
| Self-prediction fine-tuning instead of baseline fine-tuning | GPT-4o · Situational Awareness Dataset, Predict Tokens plain | score: 0.26 -> 0.41 | Helped |
| Self-prediction training | Models predicting properties that depend on long responses · Long-response behavior properties | accuracy relative to most-common-behavior baseline | Hurt |
| Self-prediction training rather than baseline | GPT-4o · Steganography | undetected accuracy: 1% absolute gain | No clear effect |
Surprising findings
- Self-prediction models fail to beat simple baselines on longer-response properties such as movie review sentiment and response length.
- No self-prediction advantage for sycophancy-type 'Are you sure?' bias detection, where cross-prediction by GPT-4o (76.5%) beats Llama 70B self-prediction (74.5%).
Limitations
- GPT-3.5 shows little or no self-prediction advantage, likely due to weaker capabilities.
- Models fail on longer-output properties and out-of-distribution self-knowledge tasks, showing limited generalization.
- Experiments rely on simple hypothetical properties without chain-of-thought; mechanism of introspection remains speculative (self-simulation).
Experimental setup
Datasets: Wikipedia completion Country sequence Color sequence Wealth-seeking (Model-Written Evaluations) Power-seeking (Model-Written Evaluations) ARC Stories sentences Animal sequence English words Myopic reward (Model-Written Evaluations) Survival instinct (Model-Written Evaluations) MMLU
Benchmarks: Held-out introspection self-prediction tasks Situational Awareness Dataset OpenAI Evals (sandbagging, Schelling point, steganography)
Metrics: accuracy mean absolute deviation (calibration) accuracy vs. most common baseline
Baselines: Cross-prediction (GPT-4o predicting Llama 70B) Most common behavior baseline Untrained models GPT-3.5
Builds on: Out-of-context reasoning LoRA Knows what it knows finetuning
Closest papers by meaning
- Training Language Models to Explain Their Own Computations (NeurIPS 2026 · similarity 0.88)
- Tell me about yourself: LLMs are aware of their learned behaviors (ICLR 2025 · similarity 0.87)
- Language Models Represent Beliefs of Self and Others (ICML 2024 · similarity 0.87)
- Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data (NeurIPS 2024 · similarity 0.87)
- Large Language Models Think Too Fast To Explore Effectively (NeurIPS 2025 · similarity 0.87)
- Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.