Research map / Large language models / Language, safety and interpretability
Fixes That Fail: Self-Defeating Improvements in Machine-Learning Systems
Ruihan Wu, Chuan Guo, Awni Hannun, Laurens van der Maaten
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train disparity model with disparity MAE (u') instead of depth MAE (u) | PSMNet disparity estimation · KITTI | disparity MAE: 1.28 -> 1.21 | Improved |
| Improved disparity model u' feeds car detector | P-RCNN car detector · KITTI car detection | AP3D: 39.91 -> 38.75 | Hurt |
| Improved disparity model u' feeds car detector | P-RCNN car detector · KITTI car detection | AP_BEV: 50.34 -> 49.30 | Hurt |
| Improved disparity model u' feeds car detector | F-PTNET car detector · KITTI car detection | AP3D: 33.64 -> 32.78 | Hurt |
| Improved disparity model u' feeds pedestrian detector | P-RCNN pedestrian detector · KITTI pedestrian detection | AP3D: 34.42 -> 27.64 | Hurt |
| Improved disparity model u' feeds pedestrian detector | P-RCNN pedestrian detector · KITTI pedestrian detection | AP_BEV: 38.51 -> 31.47 | Hurt |
| Improved disparity model u' feeds pedestrian detector | F-PTNET pedestrian detector · KITTI pedestrian detection | AP3D: 43.02 -> 43.10 | No clear effect |
| Improved disparity model u' feeds pedestrian detector | F-PTNET pedestrian detector · KITTI pedestrian detection | AP_BEV: 50.49 -> 48.29 | Hurt |
| Disparity MAE loss improves near range but hurts far range | disparity model u' · KITTI disparity estimation, 0-20 m | disparity MAE: 1.32 -> 1.21 | Improved |
| Disparity MAE loss improves near range but hurts far range | disparity model u' · KITTI disparity estimation, 40-max m | disparity MAE: 1.11 -> 1.20 | Hurt |
| Train pedestrian detector only on pedestrians within 20 m (ablation) | P-RCNN pedestrian detector with u' · KITTI pedestrian detection | AP3D: 27.64 -> 34.20 | Improved |
| Train pedestrian detector only on nearby pedestrians, u' vs u (ablation) | P-RCNN pedestrian detector · KITTI pedestrian detection | AP_BEV: 34.09 -> 38.83 | Improved |
Closest papers by meaning
- Coherence Mechanisms for Provable Self-Improvement (NeurIPS 2026 · similarity 0.83)
- Progress or Regress? Self-Improvement Reversal in Post-training (ICLR 2025 · similarity 0.83)
- Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments (NeurIPS 2024 · similarity 0.83)
- A Theoretical Understanding of Self-Correction through In-context Alignment (NeurIPS 2024 · similarity 0.82)
- Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models (ICML 2024 · similarity 0.82)
- Explorations of Self-Repair in Language Models (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
