Research map / Large language models / Agents, code and math
Human-like Category Learning by Injecting Ecological Priors from Large Language Models into Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Prompts Claude-v2 to generate ~10k ecologically valid category learning tasks, then meta-trains transformer models (ERMI) on them to explain human category learning and achieve state-of-the-art on OpenML-CC18.
Problem
Testing ecological rationality of human cognition requires ecologically valid tasks and rational models for them; prior meta-learned cognitive models relied on hand-engineered environments.
Approach
Use an LLM (Claude-v2) to generate category learning tasks whose statistics match real-world classification data, then meta-learn a transformer (ERMI) on these tasks as an ecologically rational model of human category learning.
Key results
- ERMI explains human choices best in posterior model frequency (0.43) on the Shepard-type study, versus RMC at 0.41, while MI and PFN fit best under 10% of the time.
- On OpenML-CC18 (23 tasks, 4 features, 20 splits), ERMI reaches 70.95% mean accuracy and 2.26 mean rank, versus TabPFN 70.51% and XGBoost rank 2.61.
- ERMI's exceedance probability on OpenML-CC18 is 0.66, compared with 0.24 for TabPFN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the meta-learning model on LLM-generated ecological category-learning tasks instead of a Bayesian logistic regression prior | ERMI vs. MI · Shepard category-learning study | MSE to human error rates: 0.26 -> 0.03 | Helped |
| Train the meta-learning model on LLM-generated ecological category-learning tasks rather than synthetic priors | ERMI vs. MI and PFN · Shepard replication | posterior model frequency: MI (0.10± 0.03); PFN (0.05 ± 0.02); ERMI (0.43 ± 0.05) | Helped |
| Use LLM-generated ecological category-learning tasks for meta-training rather than the Bayesian logistic regression prior | ERMI vs. MI · Unseen-stimulus generalization | Euclidean distance to human choice probabilities: 0.67 -> 0.29 | Helped |
| Train on ecological category-learning tasks rather than using TabPFN | ERMI · OpenML-CC18 | accuracy: 70.51% -> 70.95% | Helped |
| Train on ecological category-learning tasks rather than using XGBoost | ERMI · OpenML-CC18 | rank: 2.61 -> 2.26 | Helped |
| Use LLM-generated ecological tasks rather than the Bayesian logistic regression prior | ERMI vs. MI · OpenML-CC18 | accuracy: +3.5% | Helped |
Surprising findings
- Claude-v2 applied directly to the category learning task matched human difficulty ordering poorly and was fully exemplar-based after the second block, fitting humans worse than ERMI (BIC 39816.90 vs 38722.13).
- GPT-4 generated data largely consisted of uniform samples with simple heuristic rules, so its statistics resembled the synthetic MI data rather than real-world data.
Limitations
- ERMI learns faster than humans and does not capture human learning pace.
- Generation is limited to binary classification with a fixed number of features; scaling to more features and classes is uncertain.
- LLM-generated data is US/western-centric.
Experimental setup
Datasets: LLM-generated category learning tasks (Claude-v2) OpenML-CC18 synthetic Bayesian logistic regression tasks synthetic Bayesian neural network tasks human category learning data (Shepard et al. type 1-6; Nosofsky-style experiments)
Benchmarks: OpenML-CC18 Shepard-style category learning experiment (types 1-6) human generalization experiment (transfer stimuli T1-T7)
Metrics: accuracy rank exceedance probability posterior model frequency mean squared error BIC learning curve error
Baselines: MI PFN RMC GCM prototype model Rule Rulex Claude-v2 SVM XGBoost TabPFN logistic regression
Builds on: meta-learned inference prior-fitted networks TabPFN rational model of categorization Claude-v2
Closest papers by meaning
- Turning large language models into cognitive models (ICLR 2024 · similarity 0.87)
- Hypothesis, Verification, and Induction: Grounding Large Language Models with Self-Driven Skill Learning (AAAI 2024 · similarity 0.87)
- Learning to Learn with Contrastive Meta-Objective (NeurIPS 2025 · similarity 0.86)
- Any-Way Meta Learning (AAAI 2024 · similarity 0.86)
- In-Context Learning Agents Are Asymmetric Belief Updaters (ICML 2024 · similarity 0.86)
- Generating Computational Cognitive models using Large Language Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.