Research map / Federated learning and privacy / Privacy attacks and secure learning
Enhancing Privacy in the Early Detection of Sexual Predators Through Federated Learning and Differential Privacy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a federated learning pipeline with three differential privacy variants (metric DP on embeddings, DP-SGD, DP-FedAvg) for early detection of online grooming, evaluated on the PANC dataset with a BERT+logistic regression classifier.
Problem
Early detection of online grooming (sexual predators in chats) typically requires centralized access to children's private messages, raising privacy concerns, and existing grooming datasets are scarce and unrealistic.
Approach
Train the eSPD classifier with federated learning across user devices, combined with differential privacy in three placements (noise on token embeddings via metric DP, instance-level DP-SGD on clients, and user-level DP-FedAvg on the server), and compare their privacy-utility trade-offs using latency-weighted F1 under a sliding-window early-warning setup.
Key results
- The FL model (no DP) reaches F1 0.82 and F-latency 0.64 with FPR 0.03, versus 0.63 F-latency for the centralized model.
- FL with DP-FedAvg (epsilon=1, delta=1e-5) achieves F-latency 0.63 and F1 0.81, close to the non-private FL model.
- FL with DP-SGD reaches F-latency 0.61 at epsilon=1, while metric DP (eta=20) drops to F-latency 0.52.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use federated training with FedAvg instead of training centrally | BERT+LR eSPD model · PANC eSPD | F1: 0.75 -> 0.82 | Helped |
| Use federated training with FedAvg instead of training centrally | BERT+LR eSPD model · PANC eSPD | F-latency: 0.63 -> 0.64 | No clear effect |
| Use federated training with FedAvg instead of training on warm-up data only | BERT+LR eSPD model · PANC eSPD | F-latency: 0.48 -> 0.64 | Helped |
| Add metric DP to federated training (η=20) | FL model · PANC eSPD | F-latency: 0.64 -> 0.52 | Hurt |
| Use a more restrictive metric-DP privacy budget (η=10) | FL model · PANC eSPD | F-latency: 6% | Hurt |
| Add instance-level DP-SGD to federated training with (ε=1, δ=10^-5) | FL model · PANC eSPD | F-latency: 0.64 -> 0.61 | Hurt |
| Use DP-SGD with (ε=0.50, δ=10^-5) | FL model · PANC eSPD | F-latency: 57% | Hurt |
| Add user-level DP-FedAvg to federated training | FL model · PANC eSPD | F-latency: 0.64 -> 0.63 | No clear effect |
| Fine-tune BERT_mobile in a federated setting instead of centrally | BERT_mobile eSPD model · PANC eSPD | F-latency: 0.58 -> 0.58 | No clear effect |
| Set the classification threshold to achieve a 1% FPR | Centralized model · PANC eSPD | F-latency: 0.63 -> 0.59 | Hurt |
| Set the classification threshold to achieve a 1% FPR | FL model · PANC eSPD | F-latency: 0.64 -> 0.61 | Hurt |
Surprising findings
- Federated training without any DP outperformed the centralized model on F-latency (0.64 vs 0.63) and FPR (0.03 vs 0.07).
- Metric DP with eta=10 produced near-non-performing models (F-latency about 6%).
Limitations
- Training data relies on Perverted Justice chats, which differ from real-life grooming conversations.
- DP-FedAvg requires a trusted server and many clients for good utility, which is problematic when labeled data is scarce.
- Instance-level DP-SGD gives limited protection for users who contribute thousands of messages; metric DP's eta is not comparable to standard epsilon.
Experimental setup
Datasets: PANC PAN12 ChatCoder2
Benchmarks: PANC
Metrics: F1 recall precision speed F-latency FPR
Baselines: Warm-up model Centralized model Centralized fine-tuned BERT_base Centralized fine-tuned BERT_mobile
Builds on: FedAvg DP-SGD DP-FedAvg PANC dataset eSPD framework
Closest papers by meaning
- One-shot Empirical Privacy Estimation for Federated Learning (ICLR 2024 · similarity 0.84)
- Little Is Enough: Boosting Privacy by Sharing Only Hard Labels in Federated Semi-Supervised Learning (AAAI 2025 · similarity 0.83)
- A Simple and Practical Method for Reducing the Disparate Impact of Differential Privacy (AAAI 2024 · similarity 0.83)
- Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining (ICML 2024 · similarity 0.82)
- Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning (ICML 2024 · similarity 0.82)
- Federated Behavioural Planes: Explaining the Evolution of Client Behaviour in Federated Learning (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.