Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Divide-and-Conquer Predictive Coding: a structured Bayesian inference algorithm
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes divide-and-conquer predictive coding (DCPC), a particle-based predictive coding algorithm that samples each latent's complete conditional via Langevin proposals from prediction errors inside divide-and-conquer SMC, and learns generative parameters by local gradients.
Problem
Predictive coding algorithms are limited to Gaussian generative models, Laplace (MAP) posteriors, and mean-field factorization, so they perform poorly on high-dimensional structured inference and yield inaccurate parameter updates.
Approach
Use prediction errors defined as score functions of the complete conditional to parameterize preconditioned Langevin proposals for each latent coordinate, correct them by importance weighting and resampling, nest them in divide-and-conquer SMC so particles target the joint posterior, and update parameters with local particle-averaged log-joint gradients.
Key results
- On MNIST, DCPC reaches NLL 102.5 ± 0.01 versus 144.6 ± 0.7 for MCPC.
- On CelebA at 32x32 with PGD's architecture, DCPC reaches FID 82.7 ± 0.9 versus 100 ± 2.7 for PGD.
- On CelebA at 64x64, DCPC reaches FID 79.0 ± 0.9 versus 86.3 for a VAE and roughly 120 reported for LPC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DCPC instead of MCPC | Deep latent Gaussian model · MNIST held-out | NLL: $144.6 \pm 0.7$ -> $\mathbf{102.5} \pm \mathbf{0.01}$ | Helped |
| Use DCPC instead of MCPC | Deep latent Gaussian model · MNIST held-out | Mean Squared Error: $(8.29 \pm 0.05) \times 10^{-2}$ -> $\mathbf{0.01} \pm \mathbf{7.2 \times 10^{-6}}$ | Helped |
| Use DCPC instead of PGD | CelebA generator at 32 × 32 · CelebA 32 × 32 | FID: $100 \pm 2.7$ -> $\mathbf{82.7 \pm 0.9}$ | Helped |
| Use DCPC instead of LPC | CelebA generator at 64 × 64 · CelebA 64 × 64 | FID: $120$ (approximate) -> $\mathbf{79.0} \pm 0.9$ | Helped |
| Use DCPC instead of VAE | CelebA generator at 64 × 64 · CelebA 64 × 64 | FID: $86.3 \pm 0.03$ -> $\mathbf{79.0} \pm 0.9$ | Helped |
| Use DCPC instead of PGD | CelebA generator (K=10) · CelebA | FID: $100 \pm 2.7$ -> $\mathbf{82.7} \pm 0.9$ | Helped |
Surprising findings
- DCPC's log-joint on CelebA (-1.8e6) is lower than PGD's (-3.8e5) and VAE's (-4.3e5), which the authors attribute to possible differences in how the supplied code averages log-joints.
Limitations
- Longer training time than amortized end-to-end variational inference
- Greater sensitivity to learning rates
- Uses naive Langevin proposals without momentum or richer preconditioning
Experimental setup
Datasets: MNIST EMNIST Fashion MNIST CelebA
Benchmarks: MNIST EMNIST Fashion MNIST CelebA
Metrics: negative log-likelihood mean squared error FID log-joint
Baselines: MCPC LPC PGD VAE
Builds on: Monte Carlo predictive coding particle gradient descent divide-and-conquer sequential monte carlo Langevin predictive coding incremental EM
Closest papers by meaning
- Predictive Coding beyond Correlations (ICML 2024 · similarity 0.89)
- Sample as you Infer: Predictive Coding with Langevin Dynamics (ICML 2024 · similarity 0.88)
- Understanding Sample Efficiency in Predictive Coding (NeurIPS 2026 · similarity 0.88)
- Understanding and Improving Optimization in Predictive Coding Networks (AAAI 2024 · similarity 0.88)
- A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks (ICLR 2024 · similarity 0.88)
- Predictive Coding Enhances Meta-RL To Achieve Interpretable Bayes-Optimal Belief Representation Under Partial Observability (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.