Research map / Architectures and efficiency / RNNs and normalization

Augmented Lagrangian Predictive Coding

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Hebbian and predictive coding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PC-ALM, a finite-inference method-of-multipliers variant of predictive coding whose per-layer Lagrange multipliers accumulate constraint errors so weight updates align with backprop, matching BP on deep narrow residual MLPs up to depth 128.

Problem

Predictive coding (PC) is a local alternative to backpropagation but lags BP substantially in deep narrow networks, and its gradients only approximate BP at finite inference budgets.

Approach

Replace PC's quadratic-penalty inference with finite-budget dual ascent on the augmented Lagrangian: one primal gradient step then a dual update per layer, so the per-layer multiplier integrates prediction errors and converges to BP adjoints while keeping layer-local updates and PC's inference budget.

Key results

• PC-ALM closes the PC-BP gap at inference budget T=2L across width-depth sweeps (N, L in {8..128}) on Fashion-MNIST and MNIST with identity, tanh, and ReLU activations.
• In linear PC networks, PC-ALM converges to a fixed point with exact BP gradients, under a spectral-radius stability condition.
• PC-ALM reaches 0.9 cosine similarity to BP roughly an order of magnitude more efficiently than widening PC, per the paper's resource argument.

Surprising findings

• Credit signals in PC-ALM propagate ballistically and distribute evenly across layers, whereas PC's credit propagation is diffusive and stalls in very deep networks.
• Per-layer dual multipliers are unstable in some prior ADMM-style setups, yet here the augmented dual rate with a single primal step is stable under a Jury-type condition.

Limitations

• PC-ALM doubles activation memory relative to PC.
• Convergence guarantees are only proven for linear networks; nonlinear results are empirical.
• Experiments are limited to small image classification benchmarks (MNIST-family).

Experimental setup

Datasets: Fashion-MNIST MNIST

Benchmarks: Fashion-MNIST MNIST

Metrics: test accuracy cosine similarity to BP gradient

Baselines: PC BP muPC equilibrated PC

Builds on: predictive coding method of multipliers muPC equilibrium propagation ProxProp

Closest papers by meaning

• Only Strict Saddles in the Energy Landscape of Predictive Coding Networks? (NeurIPS 2024 · similarity 0.88)
• $\mu$PC: Scaling Predictive Coding to 100+ Layer Networks (NeurIPS 2025 · similarity 0.87)
• Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation (ICLR 2025 · similarity 0.87)
• Training a Predictive Coding Network on ImageNet using Equilibrium Propagation (NeurIPS 2026 · similarity 0.86)
• Sample as you Infer: Predictive Coding with Langevin Dynamics (ICML 2024 · similarity 0.86)
• Understanding Sample Efficiency in Predictive Coding (NeurIPS 2026 · similarity 0.86)