Research map / Learning theory and algorithms / Prediction and decision losses
A Zero-Positive Learning Approach for Diagnosing Software Performance Regressions
Mejbah Alam, Justin Gottschlich, Nesime Tatbul, Javier S Turek, Tim Mattson, Abdullah Muzahid
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use normal-version-only training with AutoPerf instead of the compared supervised decision-tree approach | AutoPerf versus DT · Programs in the diagnosis ability table | false negative rate: AutoPerf reports 0.0 for all anomalous programs; DT reports nonzero rates for bodytrack, histogram, linear_regression, r | Improved |
| use AutoPerf rather than UBL | AutoPerf and UBL · Programs in the diagnosis ability table | false positive rate: AutoPerf is 0.0 on seven programs; UBL reports 0.2, 0.8, 0.2, 0.4, 0.4, 0.15, and 0.6 on those programs | Improved |
| use AutoPerf rather than UBL | AutoPerf and UBL · Programs in the diagnosis ability table | false negative rate: AutoPerf is 0.0 on all ten anomalous programs; UBL has nonzero rates on bodytrack, histogram, linear_regression, reverse | Improved |
| replace function-specific or clustered autoencoders with one autoencoder across all seven functions (ablation) | AutoPerf · Synthetic seven-function program | F1 score: from 0.73 to 0.81 -> 0.31 | Hurt |
| increase the k-means cluster count from 2 to 3 to 4 (ablation) | AutoPerf · Synthetic seven-function program | F1 score | Improved |
| cluster functions rather than train one autoencoder per function (ablation) | AutoPerf · Memcached | F1 score | No clear effect |
| cluster functions rather than train one autoencoder per function (ablation) | AutoPerf · Boost and MySQL | F1 score | Hurt |
| use k-means clustering to share autoencoders across similar functions (ablation) | AutoPerf · Boost, Memcached, and MySQL | training time: ~2.5x to 5x | Improved |
| set the anomaly threshold from reconstruction-error mean and standard deviation rather than an arbitrary input-vector threshold (ablation) | AutoPerf · 10 candidate applications | accuracy: true positive rate of 1 and false positive rate of 0.05 using $\thr{\gamma}{t}$ at $t=2$ | Improved |
| use hardware performance counters for profiling | AutoPerf · Evaluated applications | profiling overhead: 4\% average | Improved |
Closest papers by meaning
- TestGenEval: A Real World Unit Test Generation and Test Completion Benchmark (ICLR 2025 · similarity 0.83)
- kGym: A Platform and Dataset to Benchmark Large Language Models on Linux Kernel Crash Resolution (NeurIPS 2024 · similarity 0.82)
- SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads? (ICML 2026 · similarity 0.82)
- Teaching Large Language Models to Self-Debug (ICLR 2024 · similarity 0.82)
- Precise Debugging Benchmark: Is Your Model Debugging or Regenerating? (NeurIPS 2026 · similarity 0.82)
- Less Is More: Adaptive Program Repair with Bug Localization and Preference Learning (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
