Research map / Large language models / AI and society

The Lock-in Hypothesis: Stagnation by Algorithm

ICML 2025 · Large language models › AI and society · approach: Data commons and human baselines · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a Bayesian formal model of human-LLM feedback loops leading to collective lock-in, then tests it with agent-based GPT-4.1-nano simulations and regression-kink analysis on WildChat conversation diversity.

Problem

Feedback loops between LLMs and human users, where models learn beliefs from human data and feed them back, may entrench existing beliefs and reduce diversity, potentially locking in false beliefs; prior evidence is mainly lab-based and lacks mechanistic explanation.

Approach

Formalize lock-in as irreversible belief entrenchment via a trust-matrix Bayesian model: when the spectral radius of the mutual-trust feedback matrix exceeds 1 (equivalently (N-1)λ1λ2>1 in the human-LLM case), agents converge confidently to a false value; validate with LLM agent simulations and measure lineage diversity on real GPT usage logs across model releases.

Key results

• Phase change at (N-1)λ1λ2=1: beyond it, every simulation run converged to a false estimate of μ, while below it runs converged to the ground truth.
• On WildChat, Hypothesis 2 (discontinuous diversity drop at GPT version releases) is supported for GPT-4-0125-preview and GPT-3.5-turbo-0613 with value-laden lineage diversity; per-user regression shows negative impact across all six metric combinations, with only two significant.
• Hypothesis 1 (downward collective diversity trend) is supported for GPT-4 on value-laden concepts but GPT-3.5-turbo shows an upward trend, so evidence is ambiguous.

Measured effects reported in the paper

Surprising findings

• GPT-3.5-turbo value-laden concept diversity trends upward over time (p<.05), contrary to the lock-in hypothesis.
• Agents in the 100-agent simulation converged to extreme views (around 0.9 or 0.1) rather than the mean, across all four topics.

Limitations

• Observational WildChat analysis cannot fully rule out temporal confounders; results described as early-stage evidence.
• Simulations are simplified (text-based beliefs, GPT-4.1-nano agents) and do not model real-world evidence incorporation or actual LLM training.
• Per-user regression finds negative effects but only two of six specifications are statistically significant.

Experimental setup

Datasets: WildChat-1M r/ChangeMyView posts

Benchmarks: WildChat-1M

Metrics: lineage diversity depth diversity topic entropy Jaccard distance semantic diversity entropy

Baselines: Shannon entropy topic entropy Jaccard distance depth diversity

Builds on: iterated learning information cascades belief propagation regression kink design

Closest papers by meaning

• Feedback Loops With Language Models Drive In-Context Reward Hacking (ICML 2024 · similarity 0.88)
• In-Context Learning Agents Are Asymmetric Belief Updaters (ICML 2024 · similarity 0.87)
• Reflection-Bench: Evaluating Epistemic Agency in Large Language Models (ICML 2025 · similarity 0.87)
• Stability Regimes for Framing-Sensitive Fine-Tuning in Language Models (NeurIPS 2026 · similarity 0.86)
• Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models (NeurIPS 2026 · similarity 0.86)
• Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond) (NeurIPS 2025 · similarity 0.86)