Research map / Large language models / Language, safety and interpretability
A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a phenomenological definition of emergence and studies it with Transformers trained on a probabilistic context-sensitive formal language, modeling the learning transition as bond percolation on a bipartite graph.
Problem
Emergence in neural networks (sudden capability gains with scale) lacks a precise definition, and its causal factors are unclear; the paper seeks a definition and a predictive account of when capabilities emerge under data scaling.
Approach
Defines emergence via three properties (sudden task improvement, simultaneous improvement across tasks, and acquisition of an underlying structure), validates it in a controlled PCSG-based language, and explains the scaling of emergence points with a bipartite-graph percolation model giving threshold proportional to sqrt(|E||K|).
Key results
- Training shows three phases: grammar acquisition (~100 iterations), relative type constraints (~1000 iterations) with sudden task improvements, and descriptive type constraints with sublinear growth.
- Emergence point for descriptive type checks and free generation scales with sqrt(|K|), with a 0.5 exponent collapse matching the percolation prediction.
- Unscrambling of descriptive sentences shows a distinct 1.5 exponent scaling, and unscrambling overall scales as |K|^{3/2}, which the authors leave unexplained.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learn the grammar underlying the formal language | GPT formal-language model · Unscrambling | per-token accuracy | Helped |
| Learn relative type constraints after grammar acquisition | GPT formal-language model · Relative type checks | accuracy: essentially zero to perfect accuracy | Helped |
| Learn grammar and relative type constraints | GPT formal-language model · Unscrambling and conditional generation | performance across reported metrics | Helped |
| Learn descriptive type constraints and generalize beyond observed entity-property pairs | GPT formal-language model · Descriptive type checks | performance: approximately 15% performance; 20% at best; ~30--35% performance | Helped |
| Increase the number of descriptive properties (ablation) | GPT formal-language model · Descriptive type checks | performance | Hurt |
| Increase the number of descriptive properties (ablation) | GPT formal-language model · Unscrambling descriptive sentences | performance and transition point | Hurt |
| Increase the number of descriptive properties (ablation) | GPT formal-language model · Relative type checks | performance | No clear effect |
| Increase the number of descriptive properties (ablation) | GPT formal-language model · Unscrambling relative sentences | performance | No clear effect |
| Increase the number of descriptive properties (ablation) | GPT formal-language model · Descriptive type checks | transition-point scaling exponent: 0.5 | Helped |
| Increase the number of descriptive properties (ablation) | GPT formal-language model · Unscrambling descriptive sentences | transition-point scaling exponent: 1.5 | Mixed |
| Increase the number of entities from 900 to 1800 (ablation) | GPT formal-language model · Unscrambling descriptive sentences | transition-point scaling exponent: from 3/2 to close to 2 | Mixed |
| Increase the number of entities from 900 to 1800 (ablation) | GPT formal-language model · Descriptive type checks | transition-point scaling exponent: 0.5 | No clear effect |
Surprising findings
- Unscrambling transition exponent (3/2) does not match the percolation prediction and changes to near 2 when |E| is doubled, which the authors leave unexplained.
Limitations
- Toy synthetic formal language rather than naturalistic data; the authors note the work is primarily demonstrative.
- Percolation derivation assumes uncorrelated bipartite graphs and no closed loops, valid only in sub-critical regime or approximation.
- Unscrambling scaling exponent is not explained by the theory.
Experimental setup
Datasets: synthetic probabilistic context-sensitive language (900 entities, 18000 properties, 10 classes)
Benchmarks: synthetic free generation, unscrambling, and conditional generation tasks
Metrics: grammaticality type check accuracy exact match per-token accuracy conditions satisfied average probability of valid tokens loss
Builds on: nanoGPT probabilistic context-free grammar bond percolation on bipartite graphs
Closest papers by meaning
- The emergence of sparse attention: impact of data distribution and benefits of repetition (NeurIPS 2025 · similarity 0.89)
- Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space (NeurIPS 2024 · similarity 0.89)
- What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation (ICML 2024 · similarity 0.88)
- Understanding Emergent Abilities of Language Models from the Loss Perspective (NeurIPS 2024 · similarity 0.87)
- Predicting Emergent Abilities with Infinite Resolution Evaluation (ICLR 2024 · similarity 0.87)
- Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.