Research map / Federated learning and privacy / Differential privacy
HiddenEcho: Mitigating Noise Amplification in Differentially Private LLMs with Hidden-State Correction
Wenhao Li, Kunhao Li, Lei Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add residual connections (ablation) | HiddenEcho · BBC News, eta=100 | AUC: 0.659 -> 0.732 | Improved |
| add hidden layer filter instead of fixed skip layers (ablation) | HiddenEcho · BBC News, eta=100 | AUC: 0.629 -> 0.732 | Improved |
| add information-bottleneck dimension reducer instead of linear layer (ablation) | HiddenEcho · Financial Phrasebank, eta=100 | AUC: 0.789 -> 0.857 | Improved |
| add hidden-state denoising correction over LDP | LDP · Financial Phrasebank, eta=100 | AUC: 0.596 -> 0.857 | Improved |
| use HiddenEcho instead of SnD pretrained denoiser | SnD · BBC News, eta=6000 | AUC: 0.637 -> 0.951 | Improved |
| use layer-filtered HiddenEcho instead of HiddenEcho-Full (ablation) | HiddenEcho-Full · BBC News, eta=100 | AUC: 0.685 -> 0.732 | Improved |
| use layer-filtered HiddenEcho instead of HiddenEcho-Full (ablation) | HiddenEcho-Full · Financial Phrasebank, eta=100 | AUC: 0.875 -> 0.857 | Hurt |
| add filtered hidden layers to reduce communication (ablation) | HiddenEcho-Full · classification | communication cost: 2.63 -> 0.38 MiB (85.55% saved) | Improved |
| add filtered hidden layers on generation tasks (ablation) | HiddenEcho-Full · IWSLT | communication cost: 6.00 -> 2.25 MiB (62.50% saved) | Improved |
| use HiddenEcho-Full instead of SnD for training speed | SnD · BBC News | time per epoch: 393 -> 118 s (72.52% reduction) | Improved |
| use HiddenEcho client correction at the final layer | LDP · hidden-state MSE, eta=100 | MSE: 14.69 -> 8.31 | Improved |
| train HiddenEcho-Full for more epochs | HiddenEcho-Full · BBC News | eval loss | Hurt |
Closest papers by meaning
- Split-and-Denoise: Protect large language model inference with local differential privacy (ICML 2024 · similarity 0.89)
- Hidden No More: Attacking and Defending Private Third-Party LLM Inference (ICML 2025 · similarity 0.87)
- DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer (ICLR 2024 · similarity 0.86)
- ScaleOT: Privacy-utility-scalable Offsite-tuning with Dynamic LayerReplace and Selective Rank Compression (AAAI 2025 · similarity 0.86)
- Scaling Laws for Differentially Private Language Models (ICML 2025 · similarity 0.86)
- SentinelLMs: Encrypted Input Adaptation and Fine-Tuning of Language Models for Private and Secure Inference (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
