Research map / Learning theory and algorithms / Matrix and tensor methods
Analysis of Deep Neural Networks with Extended Data Jacobian Matrix
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase depth from 1 to 4 layers at fixed hidden units (ablation) | feed-forward net · CIFAR-10 val | EDJM score: 5.99 -> 21.40 | Helped |
| increase depth from 1 to 4 layers at fixed hidden units (ablation) | feed-forward net · CIFAR-10 val | accuracy: 53.94% -> 56.34% | Helped |
| increase depth from 1 to 4 layers at fixed hidden units (ablation) | feed-forward net · MNIST val | accuracy: 98.53% -> 98.66% | Helped |
| replace feed-forward with convolutional network (ablation) | 4-layer feed-forward · CIFAR-10 val | accuracy: 56.34% -> 73.77% | Helped |
| replace feed-forward with convolutional network (ablation) | 4-layer feed-forward · CIFAR-10 val | EDJM score: 21.40 -> 59.65 | Helped |
| add dropout (ablation) | 2-layer feed-forward · CIFAR-10 val | accuracy: 54.23% -> 57.78% | Helped |
| add dropout (ablation) | 2-layer feed-forward · MNIST val | accuracy: 98.59% -> 98.74% | Helped |
| train student on teacher soft labels instead of hard labels (ablation) | feed-forward shallow · CIFAR-10 | accuracy: 62.3% -> 73.6% | Helped |
| train student on teacher soft labels instead of hard labels (ablation) | convolutional shallow · CIFAR-10 | accuracy: 83.1% -> 86.7% | Helped |
| train student on teacher soft labels instead of hard labels (ablation) | feed-forward shallow · CIFAR-10 | EDJM score: 33.7 -> 73.6 | Helped |
| ensemble 2-, 3-, 4-layer nets (ablation) | individual feed-forward nets · CIFAR-10 val | accuracy: 56.34% -> 57.60% | Helped |
| ensemble 2-, 3-, 4-layer nets (ablation) | individual feed-forward nets · CIFAR-10 val | EDJM score: 13.94 -> 8.75 | Hurt |
Closest papers by meaning
- An Infinite-Width Analysis on the Jacobian-Regularised Training of a Neural Network (ICML 2024 · similarity 0.87)
- Investigating the Overlooked Hessian Structure: From CNNs to LLMs (ICML 2025 · similarity 0.85)
- Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss (ICML 2024 · similarity 0.84)
- Neglected Hessian component explains mysteries in sharpness regularization (NeurIPS 2024 · similarity 0.84)
- Deep linear networks for regression are implicitly regularized towards flat minima (NeurIPS 2024 · similarity 0.84)
- Differentiable Learning of Generalized Structured Matrices for Efficient Deep Neural Networks (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.