Research map / Learning theory and algorithms / Matrix and tensor methods
M-FAC: Efficient Matrix-Free Approximations of Second-Order Information
Elias Frantar, Eldar Kurtic, Dan Alistarh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use M-FAC instead of WoodFisher for gradual pruning | MobileNetV1-STR · ImageNet, 89% sparsity | Top-1 accuracy: 63.87 -> 65.01 | Improved |
| Use M-FAC instead of WoodFisher for gradual pruning | ResNet50 · ImageNet, 95% sparsity | Top-1 accuracy: 72.12 -> 72.32 | Improved |
| Use practical M-FAC settings rather than WoodFisher | MobileNetV1 · ImageNet, 89% sparsity | accuracy: 63.9 -> 67.3 | Improved |
| Use practical M-FAC settings rather than WoodFisher | ResNet50 · ImageNet, 98% sparsity | accuracy: 65.6 -> 67.5 | Improved |
| Update remaining weights after pruning instead of leaving them unchanged (ablation) | M-FAC pruning · ResNet20 / CIFAR-10, 90% sparsity | accuracy: 11.5 -> 28.2 | Improved |
| Use approximate simultaneous OBS updates rather than solving the full linear system (ablation) | M-FAC pruning · ResNet20 / CIFAR-10, 90% sparsity | accuracy: 28.1 vs 28.2 | No clear effect |
| Increase the number of gradients used for Fisher estimation (ablation) | M-FAC one-shot pruning · ResNet50 / ImageNet | pruned accuracy | Improved |
| Increase the number of gradients used for Fisher estimation (ablation) | M-FAC one-shot pruning · MobileNetV1-STR / ImageNet | pruned accuracy | Improved |
| Increase block size while holding the gradient count fixed (ablation) | M-FAC one-shot pruning · ResNet50 / ImageNet | pruned accuracy | Mixed |
| Increase block size while holding the gradient count fixed (ablation) | M-FAC one-shot pruning · MobileNetV1-STR / ImageNet | pruned accuracy | Hurt |
| Use M-FAC instead of SGD | ResNet20 · CIFAR-10 | test accuracy: 91.78 -> 92.34 | Improved |
| Use M-FAC instead of SGD | ResNet32 · CIFAR-10 | test accuracy: 92.80 -> 92.65 | No clear effect |
Closest papers by meaning
- Sassha: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation (ICML 2025 · similarity 0.86)
- Free Decompression with Algebraic Spectral Curves (NeurIPS 2026 · similarity 0.86)
- Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning (ICML 2024 · similarity 0.86)
- Investigating the Overlooked Hessian Structure: From CNNs to LLMs (ICML 2025 · similarity 0.85)
- Advancing the Lower Bounds: an Accelerated, Stochastic, Second-order Method with Optimal Adaptation to Inexactness (ICLR 2024 · similarity 0.85)
- COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
