Research map / Vision-language and multimodal / Human and embodied modeling
Biologically Inspired Learning Model for Instructed Vision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a biologically inspired BU-TD network trained with a 'Counter-Hebb' local learning rule that matches backpropagation under weight symmetry and uses the top-down stream both for learning and for instruction-guided sub-network selection on multi-task benchmarks.
Problem
Biologically plausible learning models use top-down streams only for feedback, ignoring the top-down role in attention and task guidance; a model is needed that integrates both learning (credit assignment) and instruction-guided visual processing.
Approach
Each BU neuron has a TD counter neuron linked by mirrored connectivity; synapses update by pre-synaptic activity times the post-synaptic counter neuron's activity (Counter-Hebb). With GaLU gating, a TD pass from a task instruction selects a task-specific BU sub-network, and the same TD network propagates errors, so one TD stream serves guidance and learning, and under symmetric weights the update
Key results
- Symmetric-init Counter-Hebb matches BP on CIFAR10 (55.54 ± 0.3 vs 55.47 ± 0.3 for BP ours) and MNIST (98.34%).
- On Multi-MNIST, CH with symmetric init reaches 94.20 ± 0.30 average task accuracy, comparable to non-biological MTL methods (93.47–94.87).
- On CelebA, symmetric-weight CH reaches 89.69 ± 0.12 vs 90.22–90.99 for MTL baselines; asymmetric init drops to 79.25 ± 1.63.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| initialize BU and TD weights asymmetrically instead of symmetrically (ablation) | Counter-Hebb learning · MNIST | test accuracy (%): 98.34 ± 0.06 -> 98.17 ± 0.06 | Hurt |
| initialize BU and TD weights asymmetrically instead of symmetrically (ablation) | Counter-Hebb learning · Fashion-MNIST | test accuracy (%): 89.99 ± 0.2 -> 89.27 ± 0.1 | Hurt |
| initialize BU and TD weights asymmetrically instead of symmetrically (ablation) | Counter-Hebb learning · CIFAR10 | test accuracy (%): 55.54 ± 0.3 -> 54.28 ± 0.2 | Hurt |
| initialize BU and TD weights asymmetrically instead of symmetrically (ablation) | BU-TD guided model · Multi-MNIST | average task test accuracy (%): 94.20 ± 0.30 -> 88.92 ± 2.15 | Hurt |
| initialize BU and TD weights asymmetrically instead of symmetrically (ablation) | BU-TD guided model · CelebA | average task test accuracy (%): 89.69 ± 0.12 -> 79.25 ± 1.63 | Hurt |
| add noisy asymmetric updates after symmetric weight initialization (ablation) | Counter-Hebb learning with ResNet-18 · CIFAR10 | test accuracy (%): 72.06 ± 0.69 -> 71.98 ± 0.17 | No clear effect |
| add weight decay to asymmetric weight initialization (ablation) | Counter-Hebb learning with ResNet-18 · CIFAR10 | test accuracy (%): 61.58 ± 0.79 -> 62.05 ± 1.03 | Helped |
| use symmetric weights rather than asymmetric weights (ablation) | BU-TD guided model · CelebA | average task test accuracy (%): 79.25 ± 1.63 -> 89.51 ± 0.21 | Helped |
| use multiple decoders rather than a single shared decoder (ablation) | BU-TD guided model · CelebA | average task test accuracy (%) | No clear effect |
Surprising findings
- Asymmetric initialization degrades performance much more than gradual convergence to symmetry, suggesting initialization matters more than exact symmetry.
- In the guided Multi-MNIST setting, a single shared decoder performs similarly to task-specific decoders, indicating the TD stream guides the BU stream without task-specific parameters.
Limitations
- Performance gap with asymmetric weights grows with task and model complexity (e.g., CelebA 79.25 vs ~90).
- Propagating signed error along the TD stream requires representing positive and negative values, which is biologically questionable; on/off channels are only suggested.
- Hyperparameters were tuned mainly for backpropagation, and no validation set was used in guided experiments.
Experimental setup
Datasets: MNIST Fashion-MNIST CIFAR10 Multi-MNIST CelebA
Benchmarks: MNIST Fashion-MNIST CIFAR10 Multi-MNIST CelebA
Metrics: test accuracy average task accuracy
Baselines: BP Equilibrium Propagation Predictive Coding Feedback Alignment CIM IMTL MGDA PCGrad GradDrop Uncertainty weighting
Builds on: backpropagation Hebbian learning feedback alignment gated linear unit bias-blocking
Closest papers by meaning
- A Dual-Stream Neural Network Explains the Functional Segregation of Dorsal and Ventral Visual Pathways in Human Brains (NeurIPS 2023 · similarity 0.87)
- Wiring Up Vision: Minimizing Supervised Synaptic Updates Needed to Produce a Primate Ventral Stream (ICLR 2022 · similarity 0.86)
- Brain-like Flexible Visual Inference by Harnessing Feedback Feedforward Alignment (NeurIPS 2023 · similarity 0.86)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.86)
- Multi-View Incremental Learning with Structured Hebbian Plasticity for Enhanced Fusion Efficiency (AAAI 2025 · similarity 0.85)
- TopoNets: High performing vision and language models with brain-like topography (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.