AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

A Zero-Positive Learning Approach for Diagnosing Software Performance Regressions

Mejbah Alam, Justin Gottschlich, Nesime Tatbul, Javier S Turek, Tim Mattson, Abdullah Muzahid

NeurIPS 2019 · Learning theory and algorithms › Prediction and decision losses · approach: Active learning and subset selection · 17 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use normal-version-only training with AutoPerf instead of the compared supervised decision-tree approachAutoPerf versus DT · Programs in the diagnosis ability tablefalse negative rate: AutoPerf reports 0.0 for all anomalous programs; DT reports nonzero rates for bodytrack, histogram, linear_regression, rImproved
use AutoPerf rather than UBLAutoPerf and UBL · Programs in the diagnosis ability tablefalse positive rate: AutoPerf is 0.0 on seven programs; UBL reports 0.2, 0.8, 0.2, 0.4, 0.4, 0.15, and 0.6 on those programsImproved
use AutoPerf rather than UBLAutoPerf and UBL · Programs in the diagnosis ability tablefalse negative rate: AutoPerf is 0.0 on all ten anomalous programs; UBL has nonzero rates on bodytrack, histogram, linear_regression, reverseImproved
replace function-specific or clustered autoencoders with one autoencoder across all seven functions (ablation)AutoPerf · Synthetic seven-function programF1 score: from 0.73 to 0.81 -> 0.31Hurt
increase the k-means cluster count from 2 to 3 to 4 (ablation)AutoPerf · Synthetic seven-function programF1 scoreImproved
cluster functions rather than train one autoencoder per function (ablation)AutoPerf · MemcachedF1 scoreNo clear effect
cluster functions rather than train one autoencoder per function (ablation)AutoPerf · Boost and MySQLF1 scoreHurt
use k-means clustering to share autoencoders across similar functions (ablation)AutoPerf · Boost, Memcached, and MySQLtraining time: ~2.5x to 5xImproved
set the anomaly threshold from reconstruction-error mean and standard deviation rather than an arbitrary input-vector threshold (ablation)AutoPerf · 10 candidate applicationsaccuracy: true positive rate of 1 and false positive rate of 0.05 using $\thr{\gamma}{t}$ at $t=2$Improved
use hardware performance counters for profilingAutoPerf · Evaluated applicationsprofiling overhead: 4\% averageImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy