Research map / Federated learning and privacy / Privacy attacks and secure learning

CLIOPATRA: Extracting Private Information from LLM Insights

NeurIPS 2026 · Federated learning and privacy › Privacy attacks and secure learning · approach: Membership inference and auditing · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CLIOPATRA, a poisoning plus prompt-injection attack that inserts crafted chats into the Clio privacy-preserving insights pipeline to extract a target user's medical condition from cluster summaries.

Problem

LLM-based insights systems such as Clio protect user chats with heuristic layers (PII redaction, clustering, aggregation, LLM auditing), but it is unclear whether these layers can be broken simultaneously to leak sensitive attributes of a target user.

Approach

An external adversary crafts nested prompt-injection poison chats that embed the target's known demographics and symptoms so they cluster with the target chat, then injects the summarizer to output the disease; a confident regex variant abstains when unsure, and a stealthy variant disguises the summary as a benign COVID-19 paper title.

Key results

• Non-stealthy LLM attack extracts the target's disease from up to 65% of chats with nearly 100% precision on Clio, compared to a 22% baseline (Qwen configuration).
• Confident C-Regex attack achieves 100% precision for Gemma, LLaMA and Claude configurations, with 80-83% abstention for those.
• Clio's LLM privacy auditor flags none of the 163 successfully attacked clusters as privacy violations (rating <=2).

Surprising findings

• LLM-based privacy auditing deems 57.7% of successful-attack clusters 5/5 privacy-preserving, and even Claude Sonnet 4.5 as auditor flags none.
• Facet extraction leaks age and gender in all tested extractor models despite instructions to remove PII.

Limitations

• Clio is not publicly available; experiments replicate its design locally with open models and synthetic target chats.
• Abstention rates are high, especially for stealthy and Gemma/LLaMA/Claude configurations, so average success is limited.
• Assumes adversary knows the minimum cluster size, the disease list, and some symptoms.

Experimental setup

Datasets: WildChat NLICE

Benchmarks: Clio pipeline on WildChat with synthetic medical target chats

Metrics: attack success rate precision abstention rate % clustered audit rating

Baselines: Baseline (Claude Sonnet 4.5 disease guess from public info) Regex Attack URANIA

Builds on: Clio URANIA WildChat all-mpnet-base-v2

Closest papers by meaning

• The Canary’s Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text (ICML 2025 · similarity 0.87)
• Beyond Memorization: Violating Privacy via Inference with Large Language Models (ICLR 2024 · similarity 0.87)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.87)
• Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives (NeurIPS 2024 · similarity 0.86)
• A Synthetic Dataset for Personal Attribute Inference (NeurIPS 2024 · similarity 0.86)
• PrivAuditor: Benchmarking Data Protection Vulnerabilities in LLM Adaptation Techniques (NeurIPS 2024 · similarity 0.86)