Research map / Optimization / Optimizers
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples
Haw-Shiuan Chang, Erik Learned-Miller, Andrew McCallum
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Emphasize samples by prediction variance rather than uniform sampling | SGD training · CIFAR 10 | testing error rate (%): 62.49$\pm$0.06 -> 60.66$\pm$0.06 | Improved |
| Emphasize samples by prediction variance rather than uniform sampling | SGD training · Noisy MNIST | testing error rate (%): 0.83$\pm$0.01 -> 0.64$\pm$0.01 | Improved |
| Emphasize samples by threshold closeness rather than uniform sampling | SGD training · Noisy MNIST | testing error rate (%): 0.83$\pm$0.01 -> 0.63$\pm$0.01 | Improved |
| Emphasize samples by prediction variance rather than uniform sampling | SGD training · MNIST | testing error rate (%): 0.55$\pm$0.01 -> 0.51$\pm$0.01 | Improved |
| Weight samples by prediction variance instead of uniform scan | SGD-Scan · CoNLL 2003 | 1-F1 error rate (%): 11.62$\pm$0.04 -> 11.24$\pm$0.06 | Improved |
| Weight samples by threshold closeness instead of uniform scan | SGD-Scan · OntoNote 5.0 | 1-F1 error rate (%): 17.80$\pm$0.05 -> 17.51$\pm$0.05 | Improved |
| Weight samples by easiness rather than uniform scan | SGD-Scan · CIFAR 100, ResNet-63 | testing error rate (%): 30.70$\pm$0.06 -> 29.92$\pm$0.09 | Improved |
| Weight samples by difficulty rather than uniform scan | SGD-Scan · Noisy MNIST | testing error rate (%): 0.81$\pm$0.01 -> 0.92$\pm$0.01 | Hurt |
| Sample by difficulty rather than uniform sampling | SGD training · Noisy MNIST | testing error rate (%): 0.83$\pm$0.01 -> 1.00$\pm$0.01 | Hurt |
| Use threshold-closeness weighting instead of prediction-variance weighting (ablation) | SGD-Scan · CIFAR 10 | testing error rate (%): 60.61$\pm$0.06 vs 61.02$\pm$0.06 | No clear effect |
| Apply threshold-closeness weighting in addition to distillation | Distillation training · MNIST (distill) | testing error rate (%): 2.27$\pm$0.01 -> 2.07$\pm$0.02 | Improved |
Closest papers by meaning
- On the Convergence of Loss and Uncertainty-based Active Learning Algorithms (NeurIPS 2024 · similarity 0.89)
- Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation (NeurIPS 2024 · similarity 0.86)
- Deep Active Learning with Noise Stability (AAAI 2024 · similarity 0.86)
- Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning (ICML 2024 · similarity 0.86)
- Debiased Active Learning with Variational Gradient Rectifier (AAAI 2025 · similarity 0.86)
- Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
