Research map / Large language models / Language, safety and interpretability
Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on synthetic random legal-move data instead of championship games | Othello-GPT · Othello legal move prediction | top-1 illegal move error rate: 5.17% -> 0.01% | Helped |
| use non-linear 2-layer MLP probe instead of linear probe | board state probing of Othello-GPT · board state probing (validation) | tile state error rate: synthetic best linear ~20.4% vs nonlinear 1.7% at x^7 | Helped |
| train on skewed dataset with one opening branch removed (ablation) | Othello-GPT · legal move prediction | top-1 illegal move error rate: 0.01% -> 0.02% | No clear effect |
| layerwise activation intervention starting at layer 4 (5 layers modified) | Othello-GPT (synthetic) · intervention benchmark, natural | average errors per case: 2.68 -> 0.12 | Helped |
| layerwise activation intervention starting at layer 4 (5 layers modified) | Othello-GPT (synthetic) · intervention benchmark, unnatural | average errors per case: 2.59 -> 0.06 | Helped |
| intervene at more than 5 layers (starting shallower than layer 4) | intervention technique · intervention benchmark | average errors per case | Hurt |
| intervene only at the deepest layers | intervention technique · intervention benchmark | average errors per case | Hurt |
| add regularization on other tiles' cross-entropy with weight beta (ablation) | intervention loss · intervention benchmark, natural, L_s=5 | average errors per case: beta 0.0 gives 0.22, values range 0.19 to 0.77 across beta | No clear effect |
Closest papers by meaning
- A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language (ICLR 2025 · similarity 0.88)
- Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space (NeurIPS 2024 · similarity 0.87)
- A Causal World Model Underlying Next Token Prediction: Exploring GPT in a Controlled Environment (ICML 2025 · similarity 0.87)
- Understanding Emergent Abilities of Language Models from the Loss Perspective (NeurIPS 2024 · similarity 0.86)
- Evaluating the World Model Implicit in a Generative Model (NeurIPS 2024 · similarity 0.86)
- Emergent Communication for Numerical Concepts Generalization (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.