Research map / Optimization / Optimizers
Fault Tolerance in Iterative-Convergent Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use partial recovery instead of restoring the full checkpoint (ablation) | SCAR checkpoint recovery · All tested models and datasets | iteration cost: 12%--42% for 3/4 failures; 31%--62% for 1/2 failures; 59%--89% for 1/4 failures | Helped |
| Reduce the fraction saved per checkpoint and increase checkpoint frequency using priority selection (ablation) | SCAR prioritized checkpoints · All tested models and datasets; 1/2 parameter loss | iteration cost: 78%--95% | Helped |
| Select checkpoint parameters by largest change rather than randomly (ablation) | SCAR checkpoint selection · All tested models and datasets | iteration cost | Helped |
| Select checkpoint parameters round-robin rather than by largest change (ablation) | SCAR checkpoint selection · Models and datasets tested | iteration cost | Mixed |
| Select checkpoint parameters uniformly at random rather than by largest change (ablation) | SCAR checkpoint selection · Models and datasets tested | iteration cost | Hurt |
| Decrease the fraction saved at each priority checkpoint while increasing checkpoint frequency (ablation) | SCAR prioritized checkpoints · All tested models and datasets | iteration cost | Helped |
| Decrease the fraction saved at each round-robin checkpoint while increasing checkpoint frequency (ablation) | SCAR round-robin checkpoints · Models and datasets tested | iteration cost | Mixed |
| Decrease the fraction saved at each random checkpoint while increasing checkpoint frequency (ablation) | SCAR random checkpoints · Models and datasets tested | iteration cost | Hurt |
| Recover fewer lost parameters using partial rather than full recovery (ablation) | Checkpoint recovery · All tested models and datasets | iteration cost | Helped |
| Run SCAR checkpointing rather than traditional full checkpoint recovery | LDA on ClueWeb12 · ClueWeb12 | iterations to the same likelihood: roughly 3 iterations sooner | Helped |
Closest papers by meaning
- Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training (ICML 2024 · similarity 0.87)
- Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML (NeurIPS 2024 · similarity 0.86)
- A Theory of Fault-Tolerant Learning (ICML 2024 · similarity 0.86)
- ASAP.SGD: Instance-based Adaptiveness to Staleness in Asynchronous SGD (ICML 2022 · similarity 0.86)
- MeCeFO: Enhancing LLM Training Robustness via Fault-Tolerant Optimization (NeurIPS 2025 · similarity 0.84)
- RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.