Research map / Continual learning / Online and replay-based
H2ST: Hierarchical Two-Sample Tests for Continual Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes H2ST, a threshold-free hierarchical two-sample test on feature maps that detects OOD samples and predicts task-ids in open-world task incremental learning with replay buffers.
Problem
Continual OOD detection in open-world task incremental learning: existing OOD scores need thresholds, depend on model output quality, and cannot identify which task an ID sample belongs to.
Approach
Cascade of task-specific source-target classifiers trained online on feature maps from a TIL model, comparing new samples against replay buffer samples via a calibrated Clopper-Pearson two-sample test; early exit on ID gives task-id and reduces the number of detection operations versus flat C2ST.
Key results
- H2ST averages 86.49% F1 and 88.05% TA over seven datasets with ER, versus 56.43% and 32.74% for the best OOD baselines... (MaxLogit/MORE per Table 1 averages: MORE 64.54 F1)
- On CIFAR-10 with ER, H2ST improves average F1 by 25.92% and TA by 30.79% over the second-best method.
- With GEM, H2ST reaches 89.12% average F1 and 89.10% TA, outperforming the second-best method by 23.23% and 20.40%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use H2ST instead of MORE with ER | ER · Average across 7 datasets | F1: 64.54 -> 86.49 | Helped |
| use H2ST instead of MORE with ER | ER · Average across 7 datasets | TA: 67.01 -> 88.05 | Helped |
| use H2ST instead of MORE with GEM | GEM · Average across 7 datasets | F1: 65.89 -> 89.12 | Helped |
| use H2ST instead of MORE with GEM | GEM · Average across 7 datasets | TA: 68.70 -> 89.10 | Helped |
| increase memory size from 40 to 100 (ablation) | H2ST · CIFAR-10 | F1: 57.41 -> 78.57 | Helped |
| increase memory size from 100 to 200 (ablation) | H2ST · CIFAR-10 | F1: 78.57 -> 93.54 | Helped |
| increase memory size from 200 to 300 (ablation) | H2ST · CIFAR-10 | F1: 93.54 -> 94.41 | Helped |
| use a hierarchical architecture instead of C2ST (ablation) | H2ST · All datasets and TIL methods | F1: +11.37% | Helped |
| use a hierarchical architecture instead of C2ST (ablation) | H2ST · All datasets and TIL methods | TA: +4.20% | Helped |
| use a hierarchical architecture instead of C2ST (ablation) | H2ST · T=9 | processing time per input: 22.0 ms -> 16.7 ms | Helped |
| increase the number of OOD test samples in the data stream from 2,000 to 8,000 (ablation) | H2ST · CIFAR-10 and CIFAR-100, averaged | ACC: +7.67% | Helped |
| increase the number of OOD test samples in the data stream from 2,000 to 8,000 (ablation) | H2ST · CIFAR-10 and CIFAR-100, averaged | F1: +24.21% | Helped |
Surprising findings
- Higher OOD detection performance led to more forgetting, since more OOD samples were absorbed into training for new tasks, creating a trade-off between new-task acquisition and retention.
Limitations
- Baseline thresholds are chosen by search using known labels, which the authors note is impractical in real deployment.
- Higher OOD detection increases forgetting, a trade-off the paper does not resolve.
- Performance depends on memory size since source samples are drawn from replay buffers.
Experimental setup
Datasets: MNIST SVHN CIFAR-10 CIFAR-100 Mini-ImageNet CoRe50 Stream-51
Benchmarks: MNIST SVHN CIFAR-10 CIFAR-100 Mini-ImageNet CoRe50 Stream-51
Metrics: F1 task-id accuracy (TA) ACC forgetting (FT)
Baselines: MSP Energy score ODIN MaxLogit Gentropy FeatureNorm MORE C2ST
Builds on: classifier two-sample test sequential covariate shift detection experience replay GEM MORE
Closest papers by meaning
- Class Incremental Learning via Likelihood Ratio Based Task Prediction (ICLR 2024 · similarity 0.90)
- Adaptive Retention & Correction: Test-Time Training for Continual Learning (ICLR 2025 · similarity 0.85)
- DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection (CVPR 2025 · similarity 0.85)
- Enhancing the Power of OOD Detection via Sample-Aware Model Selection (CVPR 2024 · similarity 0.85)
- Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning (NeurIPS 2024 · similarity 0.85)
- Revisiting Generative Replay for Class Incremental Object Detection (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.