Research map / Federated learning and privacy / Differential privacy

Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy

ICML 2025 · Federated learning and privacy › Differential privacy · approach: Differentially private synthetic data · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Cape, a black-box local differential privacy mechanism that perturbs prompt tokens using a hybrid context-aware utility (client-side MLM/LM logits plus embedding distance) and equal-width bucketized exponential sampling.

Problem

Protecting sensitive user information in prompts sent to black-box LLM inference services while keeping task utility; existing DP token-replacement methods trade privacy for utility and ignore context.

Approach

Combine client-side contextual logits from a small model (BERT or GPT-2) with embedding distance into a hybrid utility, and sample via an exponential mechanism over equal-width utility buckets (then uniform within a bucket) to mitigate the long-tail probability problem in large vocabularies.

Key results

• On SST-2 with Qwen2-1.5B zero-shot, Cape (BERT) is comparable to CUSTEXT and above SANTEXT and InferDPT in privacy-utility trade-off under KNN attack (exact values only in plots).
• At eps=1 on SST-2 with KNN top-10 privacy, Cape (BERT) reaches about 94.4% privacy score with 58.14% accuracy, versus non-private 92.54% accuracy.
• Cape perturbation takes about 0.10-0.15 s per input (batch size 2) with about 1.1-1.4 GB GPU memory, versus InferDPT 0.05 s and 1.03 GB.

Measured effects reported in the paper

Surprising findings

• CUSTEXT achieves high utility and similarity at small epsilon but gives almost no empirical privacy (about 40% privacy score at eps=1 on Wikitext), because its per-token adjacency list is small.
• Cape (BERT) underperforms GPT-2 variant and baselines at small epsilon under MTI attack, attributed to vocabulary/subword mismatch with the BERT attacker, giving inflated privacy scores (over 70%) even at eps=20.

Limitations

• Formal privacy guarantee is loose: effective epsilon is about epsilon + 8 due to the bucket-size term, and the paper does not align budgets with baselines.
• Privacy is only empirically evaluated against KNN and MTI attacks; the MTI attack is weakened by vocabulary mismatch for sub-word tokenizers.
• Only sensitive tokens are perturbed; a fixed list of stopwords and punctuation is kept unperturbed, which is a fixed non-sensitive assumption.

Experimental setup

Datasets: SST-2 QNLI Wikitext-103-v1

Benchmarks: SST-2 QNLI Wikitext-103-v1

Metrics: accuracy coherence (cosine similarity) alignment (cosine similarity) KNN attack top-10 privacy score masked token inference attack Rouge-L privacy score effective mapping set size Rouge-L F1 runtime memory

Baselines: SANTEXT SANTEXT+ CUSTEXT CUSTEXT+ InferDPT

Builds on: exponential mechanism metric local differential privacy SANTEXT CUSTEXT InferDPT

Closest papers by meaning

• Split-and-Denoise: Protect large language model inference with local differential privacy (ICML 2024 · similarity 0.88)
• Beyond Memorization: Violating Privacy via Inference with Large Language Models (ICLR 2024 · similarity 0.87)
• InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy (NeurIPS 2025 · similarity 0.87)
• Hidden No More: Attacking and Defending Private Third-Party LLM Inference (ICML 2025 · similarity 0.87)
• DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer (ICLR 2024 · similarity 0.87)
• Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives (NeurIPS 2024 · similarity 0.86)