Research map / Learning theory and algorithms / Prediction and decision losses
A method to automatically discover symbolic local learning rules
Andrea Perin, Fabio Anselmi, Stephane Deny
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a method that automatically discovers symbolic, layer-specific local loss functions to replace backpropagation, by building a dictionary of local tensor monomials (inspired by Penrose diagrams) and regressing coefficients over them.
Problem
Backpropagation is the standard training algorithm for deep networks but is non-local and biologically implausible; the paper seeks local learning rules that can replace it across architectures and tasks.
Approach
Discovers per-layer local loss functions symbolically by constructing a dictionary of monomials of local tensor variables (inspired by Penrose diagram notation) and regressing coefficients weighting these monomials, rather than hand-designing or learning opaque local rules.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| learn local rules for layer 1 instead of freezing it (degree 2) | Learned local rule · Sliced MNIST classification (held-out) | final loss: 0.97 -> 0.89 | Improved |
| learned local rule versus first layer frozen to PCA | Learned local rule · Sliced MNIST classification (held-out) | final loss: 0.84 (PCA) vs 0.89 (rule) | Hurt |
| learned local rules instead of backprop | Learned local rule · Sliced MNIST classification (held-out) | accuracy (%): 71 -> 67 | Hurt |
| learned local rules for layer 1 instead of freezing it (degree 2) | Learned local rule · Sliced MNIST reconstruction (held-out) | average loss: 3.35 -> 1.90 | Improved |
| learned local rules instead of backprop | Learned local rule · Sliced MNIST reconstruction (held-out) | average loss: 1.27 -> 1.90 | Hurt |
| learned layerwise local rules (degree 1) instead of freezing first two layers | Learned local rule · MNIST self-supervised Barlow Twins (held-out) | average loss: 22.47 -> 17.11 | Improved |
| learned local rules instead of backprop | Learned local rule · MNIST self-supervised Barlow Twins (held-out) | average loss: 5.50 -> 17.11 | Hurt |
| raise neighborhood degree from 1 to 2 (ablation) | Learned local rule · Sliced MNIST classification | loss | Improved |
| raise neighborhood degree from 1 to 2 (ablation) | Learned local rule · Sliced MNIST reconstruction | loss | No clear effect |
| raise neighborhood degree from 2 to 3 (global rules) (ablation) | Learned local rule · Sliced MNIST classification and reconstruction | loss | Hurt |
| transfer degree-2 rules to a network with different layer sizes (ablation) | Learned local rule · Sliced MNIST classification (held-out) | loss | Hurt |
Closest papers by meaning
- A Semantic Loss Function for Deep Learning with Symbolic Knowledge (ICML 2018 · similarity 0.86)
- Scaling Symbolic Methods using Gradients for Neural Model Explanation (ICLR 2021 · similarity 0.85)
- Neuro-Symbolic Temporal Point Processes (ICML 2024 · similarity 0.85)
- Federated Neuro-Symbolic Learning (ICML 2024 · similarity 0.84)
- Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length (ICLR 2025 · similarity 0.84)
- Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $\mu$ Parametrization (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
