Research map / Learning theory and algorithms / Prediction and decision losses

Improving equilibrium propagation without weight symmetry through Jacobian homeostasis

ICLR 2024 · Learning theory and algorithms › Prediction and decision losses · approach: Theory of neural network sensitivity · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes bias in generalized equilibrium propagation from finite nudge and Jacobian asymmetry, removes nudge bias via Cauchy-integral holomorphic EP, and adds a Jacobian homeostatic loss that improves functional symmetry, scaling to ImageNet 32x32.

Problem

Equilibrium propagation requires weight symmetry and infinitesimal nudges for unbiased gradients, both hard to realize in physical substrates; the separate bias contributions of weight asymmetry have not been studied, and generalized EP fails on larger tasks like CIFAR-10.

Approach

Generalized holomorphic EP estimates the exact neuronal error vector via a Cauchy integral over a circular teaching signal, removing finite-nudge bias, and a homeostatic loss (Hutchinson estimate of the Jacobian's skew-symmetric norm) penalizes Jacobian asymmetry at the fixed point without enforcing weight symmetry.

Key results

• On CIFAR-10, generalized hEP with the homeostatic loss reaches 84.3% validation accuracy versus 60.4% without it, against 88.6% for the symmetric hEP baseline.
• On Fashion MNIST, Cauchy-integral estimates with N=6 points at |beta|=0.5 match the ground-truth neuronal error vector (14.3% vs 14.7% validation error).
• On ImageNet 32x32, generalized hEP with true neuronal error vectors reaches 31.4% top-1 and 55.2% top-5 versus 36.5% and 60.8% for symmetric hEP.

Measured effects reported in the paper

Surprising findings

• The homeostatic loss has no measurable effect on performance when training the asymmetric network with RBP, suggesting it only helps while not restricting model performance.
• Networks trained with RBP drive the cosine between neuronal error vectors negative in all layers except the output, whereas hEP-trained networks keep it positive.

Limitations

• Removing nudge bias requires neurons to oscillate in the complex plane, restricting use to oscillation-based learning and complicating biological interpretation.
• Relies on converging dynamics and holomorphic (complex-differentiable) vector fields; EP still requires settling to equilibrium, which is costly on digital hardware.
• A remaining gap of about 4.3% top-1 accuracy on CIFAR-10 and about 5% top-5 on CIFAR-100 and ImageNet 32x32 relative to symmetric networks.

Experimental setup

Datasets: Fashion MNIST CIFAR-10 CIFAR-100 ImageNet 32x32

Benchmarks: Fashion MNIST CIFAR-10 CIFAR-100 ImageNet 32x32

Metrics: validation error top-1 accuracy top-5 accuracy cosine similarity of neuronal error vectors

Baselines: Classic EP RBP Symmetric hEP Feedback Alignment

Builds on: equilibrium propagation holomorphic EP recurrent backpropagation deep equilibrium models

Closest papers by meaning

• Training a Predictive Coding Network on ImageNet using Equilibrium Propagation (NeurIPS 2026 · similarity 0.88)
• Two Tales of Single-Phase Contrastive Hebbian Learning (ICML 2024 · similarity 0.87)
• Efficient Target Propagation by Deriving Analytical Solution (AAAI 2024 · similarity 0.87)
• Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks (ICML 2024 · similarity 0.87)
• One-Time Soft Alignment Enables Resilient Learning without Weight Transport (NeurIPS 2026 · similarity 0.86)
• Implicit Bias of Mirror Flow for Shallow Neural Networks in Univariate Regression (ICLR 2025 · similarity 0.86)