Research map / Large language models / Reasoning and chain of thought

Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics

NeurIPS 2024 · Large language models › Reasoning and chain of thought · approach: Theory of autoregressive reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Theoretically analyzes the reversal curse via gradient-descent training dynamics of a bilinear model and one-layer transformers, attributing it to asymmetry of learned weights under cross-entropy loss, and extends the analysis to chain-of-thought intransitivity.

Problem

Why auto-regressive LLMs trained on "A → B" fail to infer the reverse "B ← A" (the reversal curse), and why chain-of-thought is needed for multi-step deduction when facts are learned separately.

Approach

Uses training-dynamics analysis (gradient flow/SGD) rather than expressivity: under cross-entropy with unconstrained optimization, the weight from A to B increases during training without increasing the weight from B to A, so the reversal curse follows from weight asymmetry (and CoT from intransitivity).

Key results

• Training-direction next-token probability is learned nearly perfectly while validation reverse-direction probability stays at roughly uniform guess (~1/M) for multi-layer GPT-2-style transformers.
• Theorem: for one-layer transformers under zero initialization, p(x3|x1,x2) on test sequences is at most 1/M while training probability tends to 1 as t grows.
• Bilinear model: reversal loss is lower-bounded by a function of training loss, so reverse-direction loss stays large even as training loss goes to zero.

Measured effects reported in the paper

Surprising findings

• Symmetric loss would drive models to learn meaningless reversed sentences, so the paper argues CE asymmetry is the practical culprit rather than a fix.

Limitations

• Analysis restricted to three-token sequences with single-token entities and relations.
• Theory covers bilinear models and one-layer transformers under assumptions (zero initialization, orthonormal embeddings); multi-layer case only empirical.
• Experiments are on synthetic data with GPT-2-style models rather than real LLMs.

Experimental setup

Datasets: synthetic A→B / B←A entity-relationship sequences

Benchmarks: synthetic reversal curse dataset

Metrics: negative log probability of reversed-direction token next-token probability

Builds on: reversal curse chain-of-thought Tian et al. 2023 one-layer transformer dynamics (reparameterized Y, Z)

Closest papers by meaning

• The Reversal Curse: LLMs trained on “A is B” fail to learn “B is A” (ICLR 2024 · similarity 0.89)
• A Theoretical Analysis of Why Masked Diffusion Models Mitigate the Reversal Curse (NeurIPS 2026 · similarity 0.89)
• Delving into the Reversal Curse: How Far Can Large Language Models Generalize? (NeurIPS 2024 · similarity 0.88)
• Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (NeurIPS 2025 · similarity 0.86)
• SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs (ICLR 2026 · similarity 0.86)
• The Tell-Tale Norm: $\ell_2$ Magnitude as a Signal for Reasoning Dynamics in Large Language Models (ICML 2026 · similarity 0.86)