Research map / Architectures and efficiency / CNN design
Multi-Bias Non-linear Activation in Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the baseline activation with MBA | CNN · CIFAR-10 test, without data augmentation | test error: 12.61% -> 6.73 % | Helped |
| Replace the baseline activation with MBA | CNN · CIFAR-100 test, without data augmentation | test error: 37.20% -> 26.14% | Helped |
| Use MBA instead of ELU | CNN with data augmentation · CIFAR-100 test | test error: 24.28% -> 24.1% | Helped |
| Use MBA instead of GenPool | CNN · SVHN test, without data augmentation | test error: 1.69% -> 1.80% | Hurt |
| Add MBA to the shallow vanilla network | Shallow network · MNIST | test error: 2.04% -> 0.83% | Helped |
| Add MBA to the deep vanilla network | Deep network · CIFAR-10 | test error: 19.25% -> 12.32% | Helped |
| Increase MBA band count from K=2 to K=4 (ablation) | MBA model #6 · CIFAR-10 validation | validation error: 8.9% -> 6.7% | Helped |
| Increase MBA band count from K=4 to K=8 (ablation) | MBA model #6 · CIFAR-10 validation | validation error: 6.7% -> 6.6% | No clear effect |
| Learn MBA biases rather than fixing them after initialization (ablation) | MBA model #6, K=4 · CIFAR-10 validation | validation error: 10.8% -> 6.7% | Helped |
| Apply MBA to conv2 and conv3 stacks instead of only conv3 (ablation) | CIFAR-10 model #3 to model #6 · CIFAR-10 test | test error: 7.2% -> 6.7% | Helped |
| Also apply MBA to the lowest convolution stack (ablation) | CIFAR-10 model #6 to model #9 · CIFAR-10 test | test error: 6.7% -> 6.8% | Hurt |
| Increase network width in MBA models from model #4 to #6 (ablation) | CIFAR-10 network · CIFAR-10 test | test error: 10.4% -> 8.2% -> 6.7% | Helped |
Closest papers by meaning
- ReLUs Are Sufficient for Learning Implicit Neural Representations (ICML 2024 · similarity 0.87)
- Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks (NeurIPS 2025 · similarity 0.86)
- Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wild (CVPR 2025 · similarity 0.85)
- From Alexnet to Transformers: Measuring the Non-linearity of Deep Neural Networks with Affine Optimal Transport (CVPR 2025 · similarity 0.84)
- DEEP NEURAL NETWORK INITIALIZATION WITH SPARSITY INDUCING ACTIVATIONS (ICLR 2024 · similarity 0.84)
- Addressing Spectral Bias of Deep Neural Networks by Multi-Grade Deep Learning (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.