Research map / Probabilistic and scientific modeling / Dynamical systems
Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a "concept space" framework and synthetic 2D-object toy datasets to study how diffusion models learn concepts, showing learning speed follows concept signal and that capabilities emerge suddenly before naive prompting can elicit them.
Problem
Understanding what determines which concepts a generative model learns, in what order, and when it acquires the ability to manipulate them, especially when capabilities are hidden from naive input prompting.
Approach
Analyze learning dynamics as trajectories in a concept space whose axes are independent concepts; show learning speed is set by concept signal (sensitivity of the data-generating process to each concept), and that sudden turns in trajectories coincide with hidden capabilities revealed by latent interventions or overprompting but not naive prompting.
Key results
- Learning speed of a concept is controlled by its concept signal: stronger color or size signal yields faster learning of that concept.
- Latent interventions and overprompting elicit OOD generation consistently around 6,000 gradient steps, while naive prompting succeeds only in some seeds near 8,000 steps, aligning with the sudden turn in trajectories.
- With 50% masked prompts, over-prompting reaches 100% accuracy after about 1,000 steps, while naive prompting fails in three of five runs even after 2,000 steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the data signal for a concept by widening its visual separation (ablation) | Variational diffusion model on synthetic OOD classes · Synthetic color and size concepts | learning speed: 80% accuracy | Helped |
| Make concept signals highly imbalanced (ablation) | Variational diffusion model on synthetic OOD classes · Synthetic OOD class 11 | OOD generalization speed | Hurt |
| Increase the percentage of masked color prompts in training (ablation) | Variational diffusion model · Synthetic blue triangle OOD class | accuracy: accuracy above 0.8 | Hurt |
| Use linear latent intervention instead of naive input prompting | Variational diffusion model · Synthetic OOD class 11 | accuracy: around as early as 6,000 gradient steps | Helped |
| Use overprompting instead of naive input prompting | Variational diffusion model · Synthetic OOD class 11 | accuracy: around as early as 6,000 gradient steps | Helped |
| Use overprompting on a model trained with 50% masked prompts | Variational diffusion model · Synthetic OOD class 11 | accuracy: 100% accuracy after approximately 1,000 gradient steps | Helped |
| Use classifier-free guidance after the capability transition (ablation) | Conditional diffusion model · Synthetic OOD generalization | OOD generalization | Helped |
| Train with underspecified prompts by masking color tokens (ablation) | Variational diffusion model · Synthetic blue triangle OOD class | OOD generalization | Hurt |
| Patch the final-checkpoint conditioning embedding module into an intermediate checkpoint (ablation) | Variational diffusion model · Synthetic OOD class 11 | accuracy | Mixed |
| Use latent interventions rather than naive prompting | CelebA diffusion model · Female and With Hat OOD class | OOD generalization: near $5\times10^5$ gradient steps | Helped |
| Increase a single concept's signal in the three-concept setting (ablation) | Variational diffusion model · Synthetic three-concept OOD classes | accuracy on that concept | Helped |
| Vary batch size, channel count, learning rate, diffusion steps, weight decay, initialization scale, or optimizer (ablation) | Variational diffusion model · Synthetic experiments | qualitative results | No clear effect |
Surprising findings
- Capabilities for OOD concept manipulation exist before naive prompting can elicit them, so standard prompt-based evaluation underestimates model competence.
- Classifier-free guidance only becomes effective after the capability transition point.
Limitations
- Compositional setup is simple; real-world concepts are often hierarchical or relational.
- Assumes concepts are linearly embedded in the vector space.
- Findings are mainly from synthetic data; CelebA results do not show full OOD generalization at 2M steps.
Experimental setup
Datasets: synthetic 2D shapes DGP (shape, color, size, background color) CelebA
Benchmarks: synthetic concept-space OOD class accuracy CelebA compositional generalization
Metrics: accuracy of concept classifier probes gradient steps to 80% accuracy 50% capability point
Baselines: naive input prompting classifier-free guidance Stable Diffusion v1.4
Builds on: variational diffusion models classifier-free guidance disentangled representation learning grokking
Closest papers by meaning
- A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language (ICLR 2025 · similarity 0.89)
- Swing-by Dynamics in Concept Learning and Compositional Generalization (ICLR 2025 · similarity 0.88)
- Understanding Emergent Abilities of Language Models from the Loss Perspective (NeurIPS 2024 · similarity 0.88)
- Nonparametric Identification of Latent Concepts (ICML 2025 · similarity 0.87)
- The emergence of sparse attention: impact of data distribution and benefits of repetition (NeurIPS 2025 · similarity 0.87)
- Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.