Research map / Architectures and efficiency / RNNs and normalization
DNNs as Layers of Cooperating Classifiers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase depth while holding width at 100 nodes | fully connected ReLU MLP · MNIST and FMNIST | test error | Helped |
| increase depth beyond the point where performance saturates | fully connected ReLU MLP · MNIST and FMNIST | test error | No clear effect |
| increase width while holding depth at 10 layers | fully connected ReLU MLP · MNIST and FMNIST | test error | Helped |
| use a narrower network (ablation) | fully connected ReLU MLP · MNIST and FMNIST | test error | Hurt |
| use sigmoid activations instead of ReLU, with nodes switched on above 0.5 (ablation) | 7-layer, 100-node MLP · FMNIST | classification accuracy | Mixed |
| introduce training noise, such as batch normalization, explicit data noise, or a non-adaptive optimizer (ablation) | ReLU MLP · classification tasks tested | layer perplexity | Hurt |
| introduce training noise, such as batch normalization, explicit data noise, or a non-adaptive optimizer (ablation) | ReLU MLP · classification tasks tested | hidden unit saturation | Hurt |
| use the combined discrete and continuous node estimates rather than either estimate alone (ablation) | node-based layer classifiers · FMNIST | classification accuracy | Mixed |
| train for additional iterations (ablation) | node-based classifiers in later layers · FMNIST | classification accuracy | Helped |
| increase network depth (ablation) | discrete, continuous, and combined node-based classifiers · FMNIST | classification accuracy | Helped |
| increase network width (ablation) | discrete, continuous, and combined node-based classifiers · FMNIST | classification accuracy | Mixed |
Closest papers by meaning
- Feature Contamination: Neural Networks Learn Uncorrelated Features and Fail to Generalize (ICML 2024 · similarity 0.87)
- Understanding Generalization through Decision Pattern Shift (NeurIPS 2026 · similarity 0.87)
- Initial Guessing Bias: How Untrained Networks Favor Some Classes (ICML 2024 · similarity 0.87)
- From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning (ICML 2025 · similarity 0.87)
- Monitoring Primitive Interactions During the Training of DNNs (AAAI 2025 · similarity 0.87)
- Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.