Research map / Learning theory and algorithms / Prediction and decision losses
ALTBI: Constructing Improved Outlier Detection Models via Optimization of Inlier-Memorization Effect
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ALTBI, an unsupervised outlier detection method that exploits the inlier-memorization effect of likelihood-based deep generative models (IWAE/GLOW) via growing mini-batches, adaptive quantile loss truncation, and within-model loss ensembling.
Problem
Unsupervised outlier detection on contaminated training data, where deep generative models eventually fit outliers as well as inliers; the aim is to exploit the early-training inlier-memorization effect more fully and cheaply.
Approach
Increase mini-batch size exponentially over updates and truncate the per-sample loss with an adaptive quantile threshold so outliers are filtered from the gradient, then average per-sample losses across many updates of a single under-fitted model instead of ensembling multiple models or selecting an update via bi-modality.
Key results
- ALTBI-I (IWAE-based) achieves the best average AUC across 57 ADBench datasets, followed by ALTBI-G (GLOW-based), both outperforming all baselines reported.
- ALTBI shows smaller standard deviations than other baselines across datasets and seeds.
- Under DP-SGD with epsilon up to 10 on 20 tabular datasets, the truncated, batch-increment variant is reported to be more robust than the variant without truncation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive loss truncation and incrementally increasing mini-batches (ablation) | ALTBI · ADBench datasets | AUC | Helped |
| Increase the truncation percentage up to 8% (ablation) | ALTBI · ADBench datasets | AUC | Helped |
| Increase the truncation percentage beyond 8% (ablation) | ALTBI · ADBench datasets | AUC | Hurt |
| Set the mini-batch increment factor to γ=1.03 (ablation) | ALTBI · ADBench datasets | AUC | Helped |
| Increase IWAE sample count K beyond 2 (ablation) | ALTBI-I · ADBench datasets | AUC | Hurt |
| Increase the learning rate up to 1e-3 (ablation) | ALTBI · ADBench datasets | AUC | Helped |
| Increase the learning rate beyond 1e-3 (ablation) | ALTBI · ADBench datasets | AUC | Hurt |
| Vary the within-model ensemble update range (ablation) | ALTBI · ADBench datasets | AUC | No clear effect |
| Increase initial mini-batch size up to 128 (ablation) | ALTBI · ADBench datasets | AUC | Helped |
| Increase initial mini-batch size beyond 128 (ablation) | ALTBI · ADBench datasets | AUC | Hurt |
| Use mini-batch increment and truncated loss under DP-SGD (ablation) | ALTBI · 20 tabular datasets with DP-SGD | AUC | Helped |
Surprising findings
- The inlier-memorization effect is clearer when training data contain fewer outliers, motivating outlier filtering during training.
- ODIM's bi-modality heuristic for choosing the optimal update often fails and sometimes selects updates where the IM effect is absent.
Limitations
- Theoretical guarantees rely on assumptions (IM assumption, PL condition, bounded gradients) that may not hold in practice.
- Hyperparameters (n0, gamma, rho, T1, T2) were tuned on the benchmark and fixed across datasets, with sensitivity shown only through ablations.
- Only likelihood-based DGMs (IWAE, GLOW) are considered; score-based models were excluded for computational reasons.
Experimental setup
Datasets: ADBench 20news Agnews Amazon IMDB Yelp CIFAR10 MNIST-C MVTec-AD SVHN MNIST FMNIST
Benchmarks: ADBench
Metrics: AUC PRAUC
Baselines: ODIM DTE ICL GOAD DROCC DeepSVDD DAGMM COPOD ECOD IForest LOF kNN
Builds on: ODIM IWAE GLOW DP-SGD
Closest papers by meaning
- ODIM: Outlier Detection via Likelihood of Under-Fitted Generative Models (ICML 2024 · similarity 0.93)
- DOS: Diverse Outlier Sampling for Out-of-Distribution Detection (ICLR 2024 · similarity 0.88)
- RODEO: Robust Outlier Detection via Exposing Adaptive Out-of-Distribution Samples (ICML 2024 · similarity 0.88)
- Diffusion-based Semantic Outlier Generation via Nuisance Awareness for Out-of-Distribution Detection (AAAI 2025 · similarity 0.88)
- How Does Unlabeled Data Provably Help Out-of-Distribution Detection? (ICLR 2024 · similarity 0.88)
- Quantile-Based Maximum Likelihood Training for Outlier Detection (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.