Research map / Optimization / Implicit bias and generalization
Orthogonal Over-Parameterized Training
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with OPT rather than standard training | MLP · MNIST (MLP-N) | testing error (%): 6.05 -> 5.11 | Helped |
| Train with OPT rather than standard training | CNN-6 · CIFAR-100 | testing error (%): 37.59 -> 33.02 | Helped |
| Train with OPT (CP) rather than standard training | 10-layer CNN · ImageNet-2012 | top-1 error (%): 44.32 -> 43.67 | Helped |
| Apply OPT (CP) to the few-shot baseline | Mini-ImageNet 5-shot Baseline · Mini-ImageNet 5-shot | accuracy (%): 62.53 ± 0.69 -> 63.27 ± 0.68 | Helped |
| Use OPT (OGD) instead of standard training | GCN · Cora | accuracy (%): 81.3 -> 82.3 | Helped |
| Keep neuron weights fixed rather than learning them (ablation) | OPT (GS) · CIFAR-100 (CNN-6) | testing error (%): 42.61 -> 33.02 | Helped |
| Use an orthogonal transformation instead of an unconstrained matrix (ablation) | over-parameterized training, CNN-6 · CIFAR-100 | testing error (%): 37.24 -> 33.02 | Helped |
| Normalize initialized neuron weights to unit norm (ablation) | OPT (GS), CNN-6 · CIFAR-100 | testing error (%): 33.02 -> 32.54 | Helped |
| Normalize initialized neuron weights to unit norm (ablation) | OPT (LS), CNN-6 · CIFAR-100 | testing error (%): 34.48 -> 32.11 | Helped |
| Refine random weights by minimizing CoMHE before training (ablation) | OPT (GS), CNN-6 · CIFAR-100 | testing error (%): 33.02 -> 32.69 | Helped |
| Refine random weights by minimizing MHE before training (ablation) | OPT (GS), CNN-6 · CIFAR-100 | testing error (%): 33.02 -> 32.99 | No clear effect |
| Reduce stochastic sampling dimension from p=d/4 to p=d/16 (ablation) | S-OPT (GS), wide CNN-9 · CIFAR-100 | testing error (%): 25.59 -> 32.52 | Hurt |
Closest papers by meaning
- Reparameterized LLM Training via Orthogonal Equivalence Transformation (NeurIPS 2025 · similarity 0.86)
- OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems (NeurIPS 2026 · similarity 0.85)
- Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks (NeurIPS 2024 · similarity 0.85)
- Improving Adaptivity via Over-Parameterization in Sequence Models (NeurIPS 2024 · similarity 0.85)
- Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit (NeurIPS 2026 · similarity 0.85)
- Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.