Research map / Learning theory and algorithms / Kernels and regression theory
GEQ: Gaussian Kernel Inspired Equilibrium Models
Mingjie Li, Yisen Wang, Zhouchen Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace linear kernel with Gaussian kernel (MOptEqs -> GEQ, 8M) (ablation) | MOptEqs · CIFAR-100 | accuracy: 75.6 -> 78.2 | Improved |
| replace linear kernel with Gaussian kernel (MOptEqs -> GEQ, 8M) | MOptEqs · CIFAR-10 | accuracy: 94.6 -> 95.6 | Improved |
| GEQ vs MOptEqs | MOptEqs · ImageNet-100 | accuracy: 81.5 -> 83.9 | Improved |
| GEQ vs MOptEqs | MOptEqs · ImageNette | accuracy: 92.4 -> 93.2 | Improved |
| small GEQ (5M) vs ResNet-18 (10M) | GEQ · CIFAR-10 | accuracy: 93.5 -> 94.8 | Improved |
| GEQ vs ResNet-50 with about half the parameters | GEQ · ImageNet-100 | accuracy: 81.7 -> 83.9 | Improved |
| GEQ 8M vs ResNet-50 23M | GEQ · CIFAR-100 | accuracy: 77.9 -> 78.2 | Improved |
| GEQ vs MOptEqs on unseen sub-classes | MOptEqs · CIFAR-100 super-class unknown accuracy | accuracy: 77.4 -> 80.1 | Improved |
| GEQ vs MOptEqs on known sub-classes | MOptEqs · CIFAR-100 super-class known accuracy | accuracy: 80.1 -> 80.9 | Improved |
| polynomial-kernel attentive module in MOptEqs (ablation) | MOptEqs · CIFAR-100 | accuracy: 75.6 -> 75.1 | Hurt |
| sigmoid-kernel attentive module in MOptEqs (ablation) | MOptEqs · CIFAR-100 | accuracy: 75.6 -> 76.1 | Improved |
| Gaussian kernel instead of sigmoid kernel (ablation) | MOptEqs · CIFAR-100 | accuracy: 76.1 -> 78.2 | Improved |
Closest papers by meaning
- Optimization Induced Equilibrium Networks: An Explicit Optimization Perspective for Understanding Equilibrium Models (TPAMI 2022 · similarity 0.89)
- Separation and Bias of Deep Equilibrium Models on Expressivity and Learning Dynamics (NeurIPS 2024 · similarity 0.87)
- Deep Equilibrium Models are Almost Equivalent to Not-so-deep Explicit Models for High-dimensional Gaussian Mixtures (ICML 2024 · similarity 0.87)
- Positive Concave Deep Equilibrium Models (ICML 2024 · similarity 0.86)
- Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective (NeurIPS 2024 · similarity 0.84)
- Exact, Tractable Gauss-Newton Optimization in Deep Reversible Architectures Reveal Poor Generalization (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
