Research map / Large language models / AI and society
Benchmarking Predictive Coding Networks -- Made Simple
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PCX, a JAX library for predictive coding networks, and a benchmark suite comparing PC variants (iPC, nudging, MCPC) against backprop across MNIST to Tiny ImageNet, showing PC competitive on shallow VGG models but lagging on deeper ones.
Problem
Predictive coding networks (PCNs) have been evaluated mostly on small-scale tasks with inconsistent architectures and implementations, and they lag behind backprop at larger scale and depth, making scalability and fair comparison hard.
Approach
A JAX-based, PyTorch-like library (PCX) with JIT compilation plus a standardized benchmark suite of models and datasets, used to evaluate standard PC, incremental PC, Monte Carlo PC, and nudging variants (positive, negative, centered) against backprop, including the first application of nudging to PC.
Key results
- Centered nudging reaches 89.47% on CIFAR-10 and 67.19% top-1 on CIFAR-100 with VGG-5, exceeding BP-CE (60.82%) on CIFAR-100 top-1 and approaching BP-SE (66.28%).
- On deeper VGG-9 and ResNet-18, PC variants lag far behind backprop (e.g., CIFAR-10 best PC ~43-62% vs BP ~92-93%).
- PCX runs one epoch of PC training on the evaluated setup in 5.34 s versus 137.58 s for the Song et al. library in the timing table, though BP remains fastest.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use iPC instead of standard PC | PCN classifier · MNIST | accuracy: 98.11^{\pm0.03} -> 98.45^{\pm0.09} | Helped |
| use iPC instead of standard PC | PCN classifier · FashionMNIST | accuracy: 89.16^{\pm0.08} -> 89.90^{\pm0.06} | Helped |
| use centered nudging instead of PC-CE | PCN classifier · CIFAR-10 | accuracy: 86.61^{\pm0.14} -> 89.47^{\pm0.13} | Helped |
| use centered nudging instead of PC-CE | PCN classifier · CIFAR-100 Top-1 | accuracy: 60.00^{\pm0.19} -> 67.19^{\pm0.24} | Helped |
| use negative nudging instead of PC-CE | PCN classifier · Tiny ImageNet Top-1 | accuracy: 41.29^{\pm0.2} -> 46.40^{\pm0.1} | Helped |
| replace PC with iPC | PC autoencoder · CIFAR-10 reconstruction | MSE (× 10^-3): 6.67^{\pm0.10} -> 5.50^{\pm0.01} | Helped |
| replace PC with iPC | PC autoencoder · CelebA reconstruction | MSE (× 10^-3): 2.35^{\pm0.12} -> 1.30^{\pm0.12} | Helped |
| use MCPC instead of VAE | MNIST generator · MNIST generation | FID: 4.19^{\pm 0.38} -> 2.53^{\pm 0.17} | Helped |
| use MCPC instead of VAE | MNIST generator · MNIST generation | inception score: 7.91^{\pm 0.03} -> 7.13^{\pm 0.10} | Hurt |
| use forward initialization rather than zero or Gaussian initialization (ablation) | PCN classifier · FashionMNIST | accuracy | Helped |
| increase the state learning rate to 1.0 rather than using a small state learning rate (ablation) | PCN classifier · FashionMNIST | accuracy | Hurt |
| use momentum with an appropriately tuned value (ablation) | VGG-7 PCN classifier · CIFAR-100 | accuracy | Helped |
Surprising findings
- PC performance decreases with depth (VGG5 > VGG7 > VGG9 > ResNet), the opposite trend from backprop.
- Adam-based training of PCNs becomes unstable for wide layers, with the stable state learning rate depending on width, unlike SGD or BP.
Limitations
- Deeper models (VGG9, ResNet-18) fail to match backprop; generalization to very deep PCNs remains an open problem.
- JIT cannot parallelize layers, so training time scales linearly with the number of layers.
- Adam instability for wide layers and reliance on forward initialization and large hyperparameter searches.
Experimental setup
Datasets: MNIST FashionMNIST CIFAR-10 CIFAR-100 CelebA Tiny ImageNet Iris
Benchmarks: MNIST FashionMNIST CIFAR-10 CIFAR-100 Tiny ImageNet CelebA
Metrics: top-1 accuracy top-5 accuracy MSE FID inception score epoch time
Baselines: backpropagation (BP-CE) backpropagation (BP-SE) incremental PC (iPC) PC (Song et al. library) VAE Eqprop
Builds on: equinox equilibrium propagation incremental predictive coding monte carlo predictive coding
Closest papers by meaning
- $\mu$PC: Scaling Predictive Coding to 100+ Layer Networks (NeurIPS 2025 · similarity 0.89)
- Understanding Sample Efficiency in Predictive Coding (NeurIPS 2026 · similarity 0.89)
- Only Strict Saddles in the Energy Landscape of Predictive Coding Networks? (NeurIPS 2024 · similarity 0.88)
- A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks (ICLR 2024 · similarity 0.88)
- Training a Predictive Coding Network on ImageNet using Equilibrium Propagation (NeurIPS 2026 · similarity 0.87)
- Understanding and Improving Optimization in Predictive Coding Networks (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.