Research map / Probabilistic and scientific modeling / Variational and generative sampling
Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a small supervised-loss weight λ in PC-Nudge | Predictive coding network · MNIST | cosine similarity to backprop gradients: above 0.999 | Helped |
| Use PC-Nudge with λ = 0.001 and rescale the learning rate | 3-layer MLP on MNIST · MNIST train | training accuracy | No clear effect |
| Reduce λ toward zero | PC-Nudge gradient estimate · MNIST | distance from backprop gradients | Helped |
| Take more inference steps away from the free-phase equilibrium | Energy-based model gradient estimate · 3-layer nonlinear ReLU network | distance from backprop gradients | Hurt |
| Reduce the distance between free and clamped equilibria | Contrastive Hebbian learning · BP gradient approximation | approximation error | Helped |
| Use weak feedback connections | Contrastive Hebbian learning · BP gradient approximation | approximation error | Helped |
| Use weak supervision by downweighting the loss | Equilibrium propagation · BP gradient approximation | approximation error | Helped |
| Subtract the first-order Taylor correction H̄xδx | Contrastive Hebbian learning · BP gradient approximation | approximation order: second order | Helped |
| Use the first activity-dynamics step at the free-phase equilibrium | Energy-based model · BP gradient computation | gradient accuracy | Helped |
| Use predictive-coding energy instead of Hopfield energy in the contrastive update | Predictive coding network · BP gradient approximation | inference phases required: single ... instead of two | Helped |
| Scale supervised loss by λ and take the infinitesimal perturbation limit | Energy-based model · BP gradient approximation | gradient accuracy | Helped |
Closest papers by meaning
- Two Tales of Single-Phase Contrastive Hebbian Learning (ICML 2024 · similarity 0.88)
- Improving equilibrium propagation without weight symmetry through Jacobian homeostasis (ICLR 2024 · similarity 0.88)
- Understanding and Improving Optimization in Predictive Coding Networks (AAAI 2024 · similarity 0.88)
- Brain-like Variational Inference (NeurIPS 2025 · similarity 0.88)
- Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization (ICLR 2024 · similarity 0.87)
- Only Strict Saddles in the Energy Landscape of Predictive Coding Networks? (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.