Research map / Learning theory and algorithms / Prediction and decision losses
Testing For Distribution Shifts with Conditional Conformal Test Martingales
Shalev Shaer, Yarin Bar, Drew Prinster, Yaniv Romano
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use fixed-reference conditional CTM instead of growing-reference standard CTM | conformal test martingale shift detector · Synthetic immediate shift N(0,1) to N(d,1), n=1000 | empirical power / detection delay | Improved |
| use conditional CTM instead of standard CTM when shift occurs late | conformal test martingale shift detector · Synthetic delayed change-point (t0 = 4000) | early detection power | Mixed |
| use conditional CTM instead of standard CTM | conformal test martingale shift detector · Synthetic gradual drift N(lambda*t, 1) | empirical power | Improved |
| use conditional CTM instead of pairwise betting | Gaussian AR(1) shift detection · Synthetic AR(1), a=0.7 | detection delay | Improved |
| use conditional CTM instead of standard CTM with reference size n >= 500 | conformal test martingale shift detector · ImageNet-C (ViT-Base entropy score, all 15 corruptions) | empirical power | Improved |
| use conditional CTM instead of standard CTM with reference size n = 100 | conformal test martingale shift detector · ImageNet-C | empirical power | Hurt |
| add DKW-based estimation-error correction to the betting function (ablation) | fixed-reference CTM · Type-I error vs reference size | type-I error rate | Improved |
| increase reference set size n (ablation) | conditional CTM · ImageNet-C | rejection-time ratio | Improved |
| clip small betting parameter to zero (ablation) | conditional CTM and standard CTM · Appendix F ablation on clipping C | power | Improved |
Closest papers by meaning
- WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales (ICML 2025 · similarity 0.87)
- Conformal Prediction under Lévy-Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations (NeurIPS 2025 · similarity 0.86)
- Not all distributional shifts are equal: Fine-grained robust conformal inference (ICML 2024 · similarity 0.86)
- Conditional Testing based on Localized Conformal $p$-values (ICLR 2025 · similarity 0.85)
- Reducing sequential change detection to sequential estimation (ICML 2024 · similarity 0.85)
- Asymptotically Log-Optimal Bayes-Assisted Confidence Sequences for Bounded Means (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
