Research map / Optimization / Implicit bias and generalization
A Unified Fast Gradient Clipping Framework for DP-SGD
Weiwei Kong, Andres Munoz medina
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Adjoint instead of ghost clipping for gradient-norm computation | DP-SGD fully-connected layer · bias dimension m and output dimension q sweep | runtime and memory | Improved |
| use Adjoint instead of ghost clipping for gradient-norm computation | DP-SGD embedding layer · query size q and vocabulary size r sweep | runtime and memory | Improved |
| use Adjoint instead of naive per-example gradients | DP-SGD small BERT · batch sizes 100 to 1600 | runtime and peak memory | Improved |
| increase bias dimension m | GhostClip · fully-connected layer | runtime | Hurt |
| increase query size q | GhostClip · embedding layer | memory | Hurt |
Closest papers by meaning
- Improved Convergence of Differential Private SGD with Gradient Clipping (ICLR 2023 · similarity 0.92)
- Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach (ICLR 2024 · similarity 0.89)
- Clipped SGD Algorithms for Performative Prediction: Tight Bounds for Stochastic Bias and Remedies (ICML 2025 · similarity 0.88)
- Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions (NeurIPS 2025 · similarity 0.88)
- To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions (ICLR 2025 · similarity 0.87)
- Subsampling is not Magic: Why Large Batch Sizes Work for Differentially Private Stochastic Optimisation (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
