Research map / Large language models / Language, safety and interpretability
Schema-learning and rebinding as mechanisms of in-context learning and emergence
Sivaramakrishnan Swaminathan, Antoine Dedieu, Rajkumar Vasudeva Raju, Murray Shanahan, Miguel Lazaro-Gredilla, Dileep George
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase overallocation ratio from 0.1 to 3.0 (ablation) | CSCG · LIALT instruction-based test | in-context accuracy: 0.00 -> 0.89 | Improved |
| increase overallocation ratio from 0.1 to 3.0 (ablation) | CSCG · LIALT example-based test | in-context accuracy: 0.00 -> 0.91 | Improved |
| run EM to convergence instead of a single EM iteration | CSCG overallocation 3.0 · LIALT example-based test | in-context accuracy: 0.91 -> 0.93 | No clear effect |
| increase clones per token from 10 to 50 (ablation) | CSCG · GINC (context length 5, 0 examples) | in-context accuracy: 0.682 -> 0.927 | Improved |
| increase clones per token from 10 to 50 (ablation) | CSCG · GINC (context length 3, 0 examples) | in-context accuracy: 0.509 -> 0.534 | Improved |
| increase clones per token from 10 to 50 (ablation) | CSCG · GINC (context length 10, 64 examples) | in-context accuracy: 0.690 -> 0.978 | Improved |
| use few-shot prompts (n=1) instead of zero-shot (n=0) at short context | CSCG with 50 clones · GINC (context length 3) | in-context accuracy: 0.534 -> 0.445 | Hurt |
| use longer context (k=10) so that the train-test gap vanishes | CSCG with 50 clones · GINC | in-context accuracy: 0.534 -> 0.975 | Improved |
| remove rebinding (infer latent states only) (ablation) | CSCG · LIALT example-based prompts | completion correctness | Hurt |
| test prompts sampled from concepts unseen in training | CSCG · GINC | in-context accuracy | Hurt |
| increase overallocation ratio from 1.0 to 3.0 (ablation) | CSCG · LIALT example-based list reverse (1 EM) | in-context accuracy: 0.12 -> 0.88 | Improved |
| increase overallocation ratio from 1.0 to 3.0 (ablation) | CSCG · LIALT instruction-based matrix roll columns (1 EM) | in-context accuracy: 0.00 -> 0.17 | Improved |
Closest papers by meaning
- In-Context Language Learning: Architectures and Algorithms (ICML 2024 · similarity 0.89)
- Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence (ICML 2025 · similarity 0.89)
- The Missing Alignment Link of In-context Learning on Sequences (ICML 2025 · similarity 0.89)
- From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When (NeurIPS 2024 · similarity 0.89)
- Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition (ICML 2025 · similarity 0.88)
- In-Context Learning through the Bayesian Prism (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
