Research map / Architectures and efficiency / Pruning and efficient tuning

Probabilistic Neural Pruning via Sparsity Evolutionary Fokker-Planck-Kolmogorov Equation

ICLR 2025 · Architectures and efficiency › Pruning and efficient tuning · approach: Sparse training from scratch

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a probabilistic view of gradual neural pruning that models the evolution of optimal-subnetwork populations under sparsity increments with a sparsity-evolutionary Fokker-Planck-Kolmogorov equation.

Problem

Gradually sparsifying a dense neural network to a target sparsity budget with minimal performance drop, i.e., tracking how the population of optimal subnetworks evolves as sparsity increases.

Approach

Analyzes the evolution of the population of optimal subnetworks under continuous sparsity increments from a thermodynamic perspective, casting it via a Fokker-Planck-Kolmogorov equation to guide gradual pruning.

Closest papers by meaning

• Pruning via Sparsity-indexed ODE: a Continuous Sparsity Viewpoint (ICML 2023 · similarity 0.93)
• The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training (ICLR 2022 · similarity 0.91)
• Pruning Deep Neural Networks from a Sparsity Perspective (ICLR 2023 · similarity 0.91)
• DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks (NeurIPS 2021 · similarity 0.89)
• Sparsest Models Elude Pruning: An Exposé of Pruning’s Current Capabilities (ICML 2024 · similarity 0.89)
• How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.89)