Research map / Large language models / Instruction and fine-tuning
Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples for Deep Learning Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes outlier gradient analysis, a Hessian-free method that flags detrimental training samples by running outlier detection (iForest, L1/L2 norm) on per-sample training gradients, and evaluates it on synthetic data, CIFAR-10N/100N, GLUE with RoBERTa, and influential-prompt detection with Llama-2-13B.
Problem
Influence functions for identifying detrimental training samples require inverting the Hessian, which is computationally expensive and ill-posed for non-convex deep models.
Approach
Observes that the per-sample gradient term of the influence score is the only sample-specific factor, so detrimental samples can be detected as outliers in gradient space with standard outlier detectors (iForest, L1/L2-norm thresholding), avoiding the Hessian entirely.
Key results
- On CIFAR-10N Worst, outlier gradient (L1) reaches 84.20% vs 82.27% cross-entropy and 83.38% for the best influence baseline (Gradient Tracing); on CIFAR-100N, outlier gradient (L2) reaches 61.40% vs 57.36% cross-entropy.
- On the two half moons synthetic task, iForest outlier gradient trimming raises MLP test accuracy from 90% to 96% and detects outliers with 96% accuracy, versus 82% for label-correction baselines.
- For influential class detection on Llama-2-13B, outlier gradient achieves AUC 1.000 and recall 1.000 on all three benchmarks, outperforming DataInf and Gradient Tracing.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply iForest to gradient space and trim detected outliers | MLP · two half moons test set | Classification Post-Trimming (%): 90.0 -> 96.0 | Helped |
| Use L1-norm thresholding instead of iForest on gradient space | Outlier Gradient · two half moons test set | Classification Post-Trimming (%): 96.0 -> 87.0 | Hurt |
| Use L2-norm thresholding instead of iForest on gradient space | Outlier Gradient · two half moons test set | Classification Post-Trimming (%): 96.0 -> 87.0 | Hurt |
| Trim detrimental samples using L2-norm gradient outliers | ResNet-34 · CIFAR-10N Aggregate | Accuracy: 90.87 -> 92.21 | Helped |
| Trim detrimental samples using L1-norm gradient outliers | ResNet-34 · CIFAR-10N Random | Accuracy: 89.17 -> 90.66 | Helped |
| Trim detrimental samples using L1-norm gradient outliers | ResNet-34 · CIFAR-10N Worst | Accuracy: 82.27 -> 84.20 | Helped |
| Trim detrimental samples using L2-norm gradient outliers | ResNet-34 · CIFAR-100N Noisy100 | Accuracy: 57.36 -> 61.40 | Helped |
| Apply outlier gradient analysis for influential class detection | Llama-2-13B-chat · Math Problems Without Reasoning | Class Detection (AUC): 0.999 ± 0.005 -> 1.000 ± 0.000 | Helped |
| Vary the number of iForest tree estimators (ablation) | Outlier Gradient (iForest), ResNet-34 · CIFAR-10N Worst | Accuracy on Test Set (%): 83.70; 84.38; 83.71; 83.72; 83.66; 83.97; 83.84; 83.42 | No clear effect |
| Set trimming budget k to 12.5% rather than 5% (ablation) | Outlier Gradient (L2), ResNet-34 · CIFAR-10N Aggregate | Accuracy: 92.21 -> 92.78 | Helped |
| Set trimming budget k to 12.5% rather than 5% (ablation) | Outlier Gradient (L2), ResNet-34 · CIFAR-10N Random | Accuracy: 90.25 -> 91.21 | Helped |
| Set trimming budget k to 2.5% rather than 5% (ablation) | Outlier Gradient (L2), ResNet-34 · CIFAR-10N Worst | Accuracy: 82.99 -> 84.48 | Helped |
Surprising findings
- Influence scores from exact Hessian-based estimation are not reliable for detrimental samples in the non-convex MLP case, while gradient-space outlier detection still isolates them.
- The simple L1/L2-norm thresholding achieved higher detection accuracy than iForest on the synthetic task but lower post-trimming accuracy.
Limitations
- Requires specifying an outlier budget k, which is a non-trivial hyperparameter; performance depends on it.
- Less useful when training data is already clean and has no outlying gradients.
- Uses only last-layer gradients or sparse random projections for high-dimensional gradient spaces.
Experimental setup
Datasets: CIFAR-10N CIFAR-100N CIFAR-10 CIFAR-100 ImageNet QNLI SST2 QQP MRPC Sentence Transformations Math Without Reasoning Math With Reasoning
Benchmarks: CIFAR-10N CIFAR-100N GLUE (QNLI, SST2, QQP, MRPC) Sentence Transformations Math Problems Without Reasoning Math Problems With Reasoning ImageNet
Metrics: test accuracy outlier detection accuracy class detection AUC class detection recall running time
Baselines: Normalized Margin Self-Confidence Confidence Entropy Exact Hessian Gradient Tracing LiSSA DataInf Self-LiSSA Self-DataInf
Builds on: influence functions DataInf LiSSA Gradient Tracing Cleanlab confidence-based label correction
Closest papers by meaning
- A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective (NeurIPS 2024 · similarity 0.88)
- Towards Robust Influence Functions with Flat Validation Minima (ICML 2025 · similarity 0.87)
- ALTBI: Constructing Improved Outlier Detection Models via Optimization of Inlier-Memorization Effect (AAAI 2025 · similarity 0.86)
- Systematic Outliers in Large Language Models (ICLR 2025 · similarity 0.86)
- RODEO: Robust Outlier Detection via Exposing Adaptive Out-of-Distribution Samples (ICML 2024 · similarity 0.86)
- DOS: Diverse Outlier Sampling for Out-of-Distribution Detection (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.