Research map / Learning theory and algorithms / Kernels and regression theory
Gradients Weights improve Regression and Classification
Samory Kpotufe, Abdeslam Boularias, Thomas Schultz, Kyoungok Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add gradient weights (squared) to k-NN regression | k-NN · Barrett 1 | normalized MSE: 0.41 ± 0.02 -> 0.21 ± 0.02 | Improved |
| add gradient weights (squared) to kernel regression | KR · Housing | normalized MSE: 0.37 ± 0.08 -> 0.21 ± 0.04 | Improved |
| add gradient weights (squared) to k-NN regression | k-NN · Telecom | normalized MSE: 0.13 ± 0.02 -> 0.34 ± 0.05 | Hurt |
| add gradient weights (squared) to kernel regression | KR · Telecom | normalized MSE: 0.30 ± 0.02 -> 0.37 ± 0.08 | Hurt |
| normalize features by training standard deviation | KR · Barrett 1 | normalized MSE: 0.98 ± 0.03 -> 0.50 ± 0.02 | Improved |
| use squared gradient weights instead of non-squared (ablation) | KR · Barrett 1 | normalized MSE: 0.38 ± 0.03 -> 0.30 ± 0.03 | Improved |
| add gradient weights (squared) to k-NN classifier | k-NN · IJCNN | error rate: 0.0629 ± 0.0038 -> 0.0482 ± 0.0060 | Improved |
| add gradient weights (squared) to k-NN classifier | k-NN · MAGIC Gamma | error rate: 0.1884 ± 0.0118 -> 0.1875 ± 0.0092 | No clear effect |
| add gradient weights (squared) to k-NN classifier | k-NN · Shuttle | error rate: 0.0031 ± 0.0011 -> 0.0019 ± 0.0008 | Improved |
| add gradient weights (squared) to epsilon-NN classifier | e-NN · Covertype | error rate: 0.28 ± 0.01 -> 0.25 ± 0.01 | Improved |
| replace GW with ReliefF weights for k-NN (ablation) | k-NN · Shuttle | error improvement over k-NN: 0.0006±0.0011 -> -0.0019±0.0024 | Hurt |
| replace GW with ReliefF weights for k-NN (ablation) | k-NN · Covertype | error improvement over k-NN: 0.0113±0.0067 -> 0.0229±0.0075 | Improved |
Closest papers by meaning
- Gradient-Weight Alignment as a Train-Time Proxy for Generalization in Classification Tasks (NeurIPS 2025 · similarity 0.88)
- Improving Generalization via Meta-Learning on Hard Samples (CVPR 2024 · similarity 0.85)
- Implicit Regularization Paths of Weighted Neural Representations (NeurIPS 2024 · similarity 0.85)
- Gradient Aligned Regression via Pairwise Losses (ICML 2025 · similarity 0.85)
- From Logistic Regression to the Perceptron Algorithm: Exploring Gradient Descent with Large Step Sizes (AAAI 2025 · similarity 0.84)
- Weight Diffusion for Future: Learn to Generalize in Non-Stationary Environments (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
