Research map / Architectures and efficiency / RNNs and normalization
Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use pHSIC Gaussian with grouping and divisive normalization instead of backprop | 3-layer fully connected network · MNIST test | accuracy: 98.6 -> 98.1 | Hurt |
| Add grouping and divisive normalization to the Gaussian pHSIC rule (ablation) | 3-layer fully connected network · MNIST test | accuracy: 94.6 -> 98.1 | Helped |
| Add grouping and divisive normalization to the cosine pHSIC rule (ablation) | 3-layer fully connected network · CIFAR10 test | accuracy: 51.1 -> 47.6 | Hurt |
| Use a wider 2x convolutional network with the Gaussian pHSIC rule (ablation) | pHSIC Gaussian + grouping · CIFAR10 test | accuracy: 86.2 -> 90.4 | Helped |
| Use a wider 2x convolutional network with the cosine pHSIC rule (ablation) | pHSIC cosine similarity + grouping · CIFAR10 test | accuracy: 89.8 -> 91.3 | Helped |
| Replace SGD with AdamW and divisive normalization with batch normalization (ablation) | pHSIC objectives · CIFAR10 test | accuracy: 89.8 -> 91.3 | Helped |
| Optimize HSIC instead of its pHSIC approximation (ablation) | Kernelized information bottleneck · Image classification experiments | accuracy | No clear effect |
| Use the original HSIC bottleneck formulation instead of the proposed objective (ablation) | Kernelized information bottleneck · Image classification experiments | accuracy | Hurt |
| Add divisive normalization to the Gaussian pHSIC rule (ablation) | Gaussian pHSIC on hard, wide tasks · CIFAR10 | accuracy | Helped |
| Add grouping and divisive normalization to the cosine similarity rule (ablation) | Cosine pHSIC · Small fully connected image benchmarks | accuracy | No clear effect |
| Use divisive normalization or another normalization method (ablation) | Learning rules other than backprop, layer-wise classification, and cosine pHSIC · CIFAR10 | accuracy | Helped |
Closest papers by meaning
- Hebbian Deep Learning Without Feedback (ICLR 2023 · similarity 0.89)
- Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry (NeurIPS 2023 · similarity 0.88)
- Rethinking Hebbian Principle: Low-Dimensional Structural Projection for Unsupervised Learning (NeurIPS 2025 · similarity 0.87)
- Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep Learning (NeurIPS 2024 · similarity 0.86)
- Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression (ICLR 2024 · similarity 0.86)
- Two Tales of Single-Phase Contrastive Hebbian Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.