Research map / Architectures and efficiency / CNN design
Dendritic Integration Inspired Artificial Neural Networks Capture Data Correlation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes quadratic neurons with channel-wise quadratic integration (Dit-CNNs) that provably capture data covariance and improves ResNet and ConvNeXt on CIFAR and ImageNet-1K with minimal parameter overhead.
Problem
Standard neurons use linear integration plus nonlinearity, which may not capture input correlations in structured data and can require many training samples; the paper asks whether dendrite-inspired quadratic integration improves generalization and CNN accuracy.
Approach
Quadratic neurons f(x)=x^T A x + w·x + b whose expected quadratic term encodes covariance (tr(AΣ)); integrated channel-wise into a single CNN layer (Dit-CNN), following the biological dendritic quadratic integration rule, rather than pixel-wise as in prior quadratic convolutions.
Key results
- Dit-ResNet-110 reaches 94.33% on CIFAR-10 and 72.40% on CIFAR-100 with 1.76M parameters, outperforming ResNet-110 (93.57%, 70.84%) and QResNet-110 (93.88%, 71.58%).
- Dit-ConvNeXt-T achieves 82.6% top-1 on ImageNet-1K vs. ConvNeXt-T 82.1%, with 29M parameters and 5.0G FLOPs; Dit-ConvNeXt-B reaches 84.2% vs. ConvNeXt-B 83.8%.
- Removing the covariance term tr(AΣ) from Dit-ResNet-32 drops CIFAR-10 accuracy from 93.17% to 37.78%, and on ImageNet-1K Dit-ConvNeXt-T drops from 82.6% to 73.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace one ResNet layer with channel-wise quadratic neurons | ResNet-20 · CIFAR-10 | accuracy: 91.25% -> 92.66% | Helped |
| Replace one ResNet layer with channel-wise quadratic neurons | ResNet-32 · CIFAR-100 | accuracy: 68.52±0.55% -> 69.68±0.32% | Helped |
| Replace one ConvNeXt layer with channel-wise quadratic neurons | ConvNeXt-T · ImageNet-1K | top-1 accuracy: 82.1 -> 82.6 | Helped |
| Replace one ConvNeXt layer with channel-wise quadratic neurons | ConvNeXt-S · ImageNet-1K | top-1 accuracy: 83.1 -> 83.6 | Helped |
| Replace one ConvNeXt layer with channel-wise quadratic neurons | ConvNeXt-B · ImageNet-1K | top-1 accuracy: 83.8 -> 84.2 | Helped |
| Omit covariance term tr(AΣ) (ablation) | Dit-ResNet-32 · CIFAR-10 | accuracy: 93.17% -> 37.78% | Hurt |
| Omit quadratic term x^T Ax (ablation) | Dit-ResNet-32 · CIFAR-10 | accuracy: 93.17% -> 12.11% | Hurt |
| Omit covariance term tr(AΣ) (ablation) | Dit-ResNet-32 · CIFAR-100 | accuracy: 69.68% -> 22.33% | Hurt |
| Omit quadratic term x^T Ax (ablation) | Dit-ResNet-32 · CIFAR-100 | accuracy: 69.68% -> 1.32% | Hurt |
| Omit covariance term tr(AΣ) (ablation) | Dit-ConvNeXt-T · ImageNet-1K | top-1 accuracy: 82.6% -> 73.7% | Hurt |
| Omit quadratic term x^T Ax (ablation) | Dit-ConvNeXt-T · ImageNet-1K | top-1 accuracy: 82.6% -> 70.6% | Hurt |
| Use pixel-wise rather than channel-wise quadratic neurons (ablation) | ConvNeXt-T · ImageNet-1K | top-1 accuracy: 82.6 -> 82.2 | Hurt |
Surprising findings
- Removing the quadratic term at inference causes very large accuracy drops (e.g., Dit-ResNet-32 on CIFAR-100 from 69.68% to 1.32%), indicating heavy reliance on the quadratic term.
- Channel-wise quadratic integration outperforms pixel-wise quadratic convolution on ImageNet-1K (e.g., 82.6% vs 82.2% for ConvNeXt-T).
Limitations
- Theoretical analysis assumes normally distributed data, a simplification of real-world data.
- Quadratic neurons increase FLOPs, though parameter increase is small.
- No error bars for ImageNet results and CIFAR-10 reports the best of ten runs.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-1K MNIST Arabic MNIST
Benchmarks: CIFAR-10 CIFAR-100 ImageNet-1K MNIST Arabic MNIST
Metrics: top-1 accuracy test accuracy parameter count FLOPs
Baselines: ResNet-20 ResNet-32 ResNet-56 ResNet-110 QResNet QuadraResNet Swin-T VMamba-T SLaK-T QuadraNet36 DeepMAD ConvNeXt-T
Builds on: ResNet ConvNeXt QResNet QuadraNet
Closest papers by meaning
- Improved Expressivity Through Dendritic Neural Networks (NeurIPS 2018 · similarity 0.86)
- Dendritic Localized Learning: Toward Biologically Plausible Algorithm (ICML 2025 · similarity 0.85)
- Beyond Linear Processing: Dendritic Bilinear Integration in Spiking Neural Networks (ICLR 2026 · similarity 0.85)
- The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks. (ICLR 2024 · similarity 0.84)
- Improving Deep Learning Speed and Performance Through Synaptic Neural Balance (AAAI 2025 · similarity 0.83)
- Convolution Goes Higher-Order: A Biologically Inspired Mechanism Empowers Image Classification (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.