Research map / Architectures and efficiency / Pruning and efficient tuning
DPaI: Differentiable Pruning at Initialization with Node-Path Balance Principle
Lichuan Xiang, Quan Nguyen-Tri, Lan-Cuong Nguyen, Hoang Pham, Khoat Than, Long Tran-Thanh, Hongkai Wen
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DPaI, a differentiable pruning-at-initialization method guided by a node-path balance principle, for pruning weights before training.
Problem
Pruning at initialization (PaI) removes weights before training to cut training and inference cost, but relies on weight saliency in an untrained network; the abstract provides limited detail on the specific failure modes addressed.
Approach
Introduces a differentiable framework for pruning at initialization guided by a node-path balance principle, applied to saliency of weights in the untrained network.
Experimental setup
Builds on: pruning at initialization
Closest papers by meaning
- Towards Data-Agnostic Pruning At Initialization: What Makes a Good Sparse Mask? (NeurIPS 2023 · similarity 0.91)
- REVISITING PRUNING AT INITIALIZATION THROUGH THE LENS OF RAMANUJAN GRAPH (ICLR 2023 · similarity 0.89)
- DHP: Differentiable Meta Pruning via HyperNetworks (ECCV 2020 · similarity 0.86)
- Pruning Neural Networks at Initialization: Why Are We Missing the Mark? (ICLR 2021 · similarity 0.86)
- DMCP: Differentiable Markov Channel Pruning for Neural Networks (CVPR 2020 · similarity 0.85)
- PDP: Parameter-free Differentiable Pruning is All You Need (NeurIPS 2023 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
