Research map / Architectures and efficiency / RNNs and normalization

Training a Predictive Coding Network on ImageNet using Equilibrium Propagation

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Scalable approximation and equivariant compute · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains a 10-layer convolutional predictive coding network (VGG10) on full-size ImageNet using equilibrium propagation with nudging and the centered scheme, reaching 13.23% top-5 error versus 12.20% for backprop.

Problem

Equilibrium propagation (EP) has only been applied to small-scale problems (e.g., ImageNet 32x32 with Hopfield networks), and predictive coding networks (PCNs) have not been trained at ImageNet scale or with EP at that scale.

Approach

Apply EP to PCNs by defining a total energy with a nudging term, using the centered finite-difference scheme and a novel asynchronous, modified projected gradient descent (mod-PGD) equilibration for the nudge phase, and compare nudging vs clamping and forward/backward/centered/random schemes.

Key results

• VGG10 trained with EP (centered scheme) on full-size ImageNet reaches 33.81% top-1 and 13.23% top-5 test error, vs 32.35% / 12.20% for the BP baseline.
• EP with random scheme reaches 34.73% top-1 and 14.02% top-5 on full-size ImageNet, using a single equilibrium state per update.
• On ImageNet 32x32 with VGG5, CE-nudge with centered scheme outperforms clamping and MSE-nudge, and centered EP is comparable to BP across MNIST, CIFAR-10, CIFAR-100 and ImageNet 32x32.

Surprising findings

• Nudge-phase equilibrium often has higher total energy than the free equilibrium, so it is not the global minimum of the total energy; PGD acts as a fixed-point iteration finding critical points.
• The random and backward schemes perform comparably to the centered scheme on ImageNet 32x32, challenging prior conclusions.

Limitations

• Gap of about 1 percentage point in top-5 error versus backpropagation on full-size ImageNet.
• Nudge-phase equilibration requires iterative PGD (K=10 iterations), which is computationally expensive compared to a single forward pass.
• No batch normalization; ResNet-like skip connection variant performs slightly worse than BP (13.32% vs 12.1% top-5).

Experimental setup

Datasets: ImageNet ImageNet 32x32 CIFAR-10 CIFAR-100 MNIST

Benchmarks: ImageNet ImageNet 32x32 CIFAR-10 CIFAR-100 MNIST

Metrics: top-1 error rate top-5 error rate

Baselines: backpropagation clamping-based PCN training AlexNet

Builds on: Equilibrium Propagation predictive coding networks Kaiming uniform initialization

Closest papers by meaning

• $\mu$PC: Scaling Predictive Coding to 100+ Layer Networks (NeurIPS 2025 · similarity 0.89)
• Only Strict Saddles in the Energy Landscape of Predictive Coding Networks? (NeurIPS 2024 · similarity 0.89)
• Improving equilibrium propagation without weight symmetry through Jacobian homeostasis (ICLR 2024 · similarity 0.88)
• Towards training digitally-tied analog blocks via hybrid gradient computation (NeurIPS 2024 · similarity 0.88)
• Benchmarking Predictive Coding Networks -- Made Simple (ICLR 2025 · similarity 0.87)
• A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks (ICLR 2024 · similarity 0.87)