AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

PAC Guarantees and Effective Algorithms for Detecting Novel Categories

Si Liu, Risheek Garrepalli, Dan Hendrycks, Alan Fern, Debashis Mondal, Thomas G. Dietterich

JMLR 2022 · Learning theory and algorithms › Prediction and decision losses · approach: Adaptive testing and experiments

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add isotonization and clipping of the estimated alien CDF (ablation)Algorithm 1 · Synthetic 9-D normal, alpha=0.01, n=100recall: 0.710±0.033 -> 0.929±0.029Improved
add isotonization and clipping of the estimated alien CDF (ablation)Algorithm 1 · Synthetic 9-D normal, alpha=0.01, n=100false positive rate: 0.033±0.027 -> 0.512±0.080Hurt
add isotonization and clipping of the estimated alien CDF (ablation)Algorithm 1 · Synthetic 9-D normal, alpha=0.5, n=10000false positive rate: 0.036±0.001 -> 0.047±0.016Hurt
add isotonization and clipping of the estimated alien CDF (ablation)Algorithm 1 · Synthetic and benchmark dataoverall performanceNo clear effect
increase sample size n from 100 to 10000 (ablation)Algorithm 1 basic CDF · Synthetic 9-D normal, alpha=0.01recall: 0.710±0.033 -> 0.932±0.009Improved
increase alien proportion alpha from 0.01 to 0.5 (ablation)Algorithm 1 basic CDF · Synthetic 9-D normal, n=100recall: 0.710±0.033 -> 0.970±0.005Improved
increase alien proportion alpha from 0.01 to 0.5 (ablation)Algorithm 1 basic CDF · Synthetic 9-D normal, n=100false positive rate: 0.033±0.027 -> 0.156±0.036Hurt
overestimate alpha with alpha' = alpha + xi (ablation)Algorithm 1 · UCI datasetsfalse positive rate: 0 to 0.246Hurt
overestimate alpha with alpha' = alpha + xi (ablation)Algorithm 1 · UCI datasetsrecall: 0.001 to 0.047Improved
estimate alpha from anomaly scores before RKHS embedding instead of raw data (ablation)KM1/KM2 alpha estimators · Synthetic 9-D normalalpha estimate biasImproved
use SPY estimator for alpha instead of bt-patrasen, C-PS or C-ROC (ablation)Algorithm 1 with alpha estimation · UCI datasetsfalse positive rateHurt
use iForest scores on covertype for bt-patrasen alpha estimate (ablation)bt-patrasen · covertypealpha estimateHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy