Research map / Optimization / Implicit bias and generalization
Manifold Neural Network with Non-Gradient Optimization
Rui Zhang, Ziheng Jiao, Hongyuan Zhang, Xuelong Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace gradient-trained DNN with ridge-regression reconstructed network (ablation) | DNN with softmax · AT&T | accuracy: 50.89 -> 90.86 | Improved |
| replace gradient-trained DNN with ridge-regression reconstructed network (ablation) | DNN with softmax · UMIST | accuracy: 72.83 -> 93.91 | Improved |
| replace gradient-trained DNN with ridge-regression reconstructed network (ablation) | DNN with softmax · WAVEFORM | accuracy: 83.98 -> 68.49 | Hurt |
| add flexible Stiefel manifold SVM decision layer (ablation) | RidgeR-NN · AT&T | accuracy: 90.86 -> 94.17 | Improved |
| add flexible Stiefel manifold SVM decision layer (ablation) | RidgeR-NN · WAVEFORM | accuracy: 68.49 -> 84.26 | Improved |
| add flexible Stiefel manifold SVM decision layer (ablation) | RidgeR-NN · UMIST | accuracy: 93.91 -> 97.39 | Improved |
| add adaptive sample weights to the SVM decision layer (ablation) | RidgeR-SVM-NN · AT&T | accuracy: 94.17 -> 98.75 | Improved |
| add adaptive sample weights to the SVM decision layer (ablation) | RidgeR-SVM-NN · WAVEFORM | accuracy: 84.26 -> 85.44 | Improved |
| add adaptive sample weights to the SVM decision layer (ablation) | RidgeR-SVM-NN · UMIST | accuracy: 97.39 -> 97.39 | No clear effect |
| use non-gradient closed-form training instead of gradient descent | DNN · UMIST and MNIST-Mini | convergence iterations: within 5 iterations vs more than 1000 | Improved |
| increase lambda above 2^0 | manifold neural network · AT&T and UMIST | accuracy and F1 | Hurt |
| vary lambda below 2^0 | manifold neural network · AT&T and UMIST | accuracy and F1 | No clear effect |
Closest papers by meaning
- ManifoldNet: A Deep Neural Network for Manifold-Valued Data With Applications (TPAMI 2020 · similarity 0.87)
- Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs (AAAI 2022 · similarity 0.86)
- Learning to Guide Random Search (ICLR 2020 · similarity 0.86)
- Deep Networks and the Multiple Manifold Problem (ICLR 2021 · similarity 0.86)
- Hardness of Learning Neural Networks under the Manifold Hypothesis (NeurIPS 2024 · similarity 0.84)
- Building Transformation Layers for Riemannian Neural Networks (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
