Research map / Federated learning and privacy / Differential privacy
Differentially Private Relational Learning with Entity-level Privacy Guarantees
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an entity-level DP-SGD variant for relational learning, combining frequency-based adaptive gradient clipping, node-without-replacement negative sampling, and a privacy amplification bound for cardinality-dependent coupled sampling, and fine-tunes BERT and Llama2 text encoders on text-attributed graphs.
Problem
Applying DP-SGD to relational learning with entity-level privacy is hard because entities appear in many edge tuples (high gradient sensitivity) and mini-batches come from coupled, multi-stage sampling that breaks standard amplification-by-subsampling analyses.
Approach
Frequency-based adaptive clipping that scales each edge tuple's clipping threshold by the maximum occurrence count of its nodes, combined with negative sampling without replacement so each node appears at most once as a negative, which makes the joint sensitivity constant; plus an RDP amplification bound for Poisson positive sampling coupled with WOR negative sampling through sample sizes only.
Key results
- Under node-level DP with epsilon=4, the proposed method fine-tuned BERT.base reaches PREC@1 of 28.00 on USA->CHN versus 26.13 for Llama2-7B and 15.18 on CHN->USA, outperforming standard clipping at the same budget (e.g., BERT.base 11.03 PREC@1 on CHN->USA with standard clipping at epsilon=4).
- Numerically, the proposed amplification bound gives substantially smaller composed DP epsilon than the naive Gaussian bound and the amplification bound with standard clipping.
- Fine-tuning with the proposed method beats the base models without fine-tuning on all four test domains even at epsilon=4 and 10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use frequency-based adaptive clipping instead of standard clipping | Llama2-7B private relational learning (ε=10) · MAG(CHN→USA) | PREC@1: 11.90 -> 18.23 | Helped |
| Use frequency-based adaptive clipping instead of standard clipping | Llama2-7B private relational learning (ε=10) · MAG(USA→CHN) | PREC@1: 20.10 -> 26.15 | Helped |
| Use frequency-based adaptive clipping instead of standard clipping | Llama2-7B private relational learning (ε=10) · AMAZ(Sports→Cloth) | PREC@1: 29.99 -> 34.82 | Helped |
| Use frequency-based adaptive clipping instead of standard clipping | Llama2-7B private relational learning (ε=10) · AMAZ(Cloth→Sports) | PREC@1: 18.14 -> 22.96 | Helped |
| Increase the capped node degree K (ablation) | Proposed private relational learning method · Text-attributed graph relation prediction | utility | Helped |
| Increase the number of negative samples per positive k_neg (ablation) | Proposed private relational learning method · Text-attributed graph relation prediction | utility | Helped |
Limitations
- Amplification analysis is limited to cardinality-dependent coupled sampling (Poisson positive sampling with WOR negative sampling); other coupled schemes are left to future work.
- Adaptive clipping penalizes gradients of high-degree nodes, which may hurt utility in some settings.
- Node degree capping at K is required for the bound to hold, and the bound depends on K.
Experimental setup
Datasets: MAG-CHN MAG-USA AMAZ-Sports AMAZ-Cloth
Benchmarks: MAG-CHN MAG-USA AMAZ-Sports AMAZ-Cloth
Metrics: PREC@1 MRR
Baselines: standard per-sample gradient clipping DP-SGD naive Gaussian mechanism bound non-private fine-tuning base models without fine-tuning
Builds on: DP-SGD privacy amplification by subsampling conditional coupling node degree capping
Closest papers by meaning
- Privacy Amplification for Matrix Mechanisms (ICLR 2024 · similarity 0.87)
- Differentially Private Prototypes for Imbalanced Transfer Learning (AAAI 2025 · similarity 0.87)
- Learning from End User Data with Shuffled Differential Privacy over Kernel Densities (ICLR 2025 · similarity 0.86)
- Efficiently Computing Similarities to Private Datasets (ICLR 2024 · similarity 0.86)
- Beyond the Calibration Point: Mechanism Comparison in Differential Privacy (ICML 2024 · similarity 0.86)
- Certification for Differentially Private Prediction in Gradient-Based Training (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.