Research map / Multi-agent systems and games / LLM agents

PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action

NeurIPS 2024 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent collaboration and safety · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PrivacyLens, a procedural framework that expands contextual-integrity privacy seeds into vignettes and simulated agent trajectories to measure unintentional privacy leakage of LM agents in action, showing strong probing accuracy but frequent leakage in final actions.

Problem

LMs acting as agents in personalized communication may unintentionally leak privacy-sensitive information accessed via tools, and existing evaluations rely on probing questions that do not reflect actual agent behavior.

Approach

Builds privacy-sensitive seeds from regulations, literature, and crowdsourcing under the contextual integrity 5-tuple, expands them into vignettes and ToolEmu-based sandbox trajectories, and evaluates the final action for leakage rate alongside probing QA, with a surgery-kit LM refinement loop and dynamic expansion of seeds into multiple trajectories.

Key results

• GPT-4 leaks sensitive information in 25.68% of cases (adjusted LR_h) under the privacy-enhancing prompt, despite answering ~99% of probing questions correctly.
• Llama-3-70B-Instruct leaks in 38.69% of cases (LR_h) with the privacy-enhancing prompt; Claude-3-Sonnet leaks in 38.83% under the basic prompt.
• Larger models improve probing accuracy but not action-based leakage: Llama-3-8B-Instruct has LR 21.30% vs Llama-3-70B-Instruct 36.51% under the basic prompt.

Measured effects reported in the paper

Surprising findings

• Strong models answer nearly all probing questions correctly yet leak sensitive information in a substantial fraction of their final agent actions.
• Scaling model size and privacy-enhancing prompting do not reduce action-based leakage, and the lowest-leakage model (Llama-3-8B) scores low on helpfulness.

Limitations

• Covers only general U.S. privacy norms and may not reflect cultural or individual differences.
• Trajectories are generated by a GPT-4 agent and only the final action is run by evaluated models, which may bias evaluation of other LMs.
• Scope limited to LM-mediated communication; other agent settings such as web agents are not studied.

Experimental setup

Datasets: ConfAIde CultureBank U.S. privacy regulations (HIPAA, FERPA, GLBA, COPPA, etc.) crowdsourced seeds via Prolific

Benchmarks: PrivacyLens ConfAIde CultureBank

Metrics: accuracy leakage rate adjusted leakage rate helpfulness score

Baselines: ConfAIde ToolEmu

Builds on: ToolEmu ReAct ConfAIde Contextual Integrity

Closest papers by meaning

• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.90)
• Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark (ICLR 2026 · similarity 0.89)
• Privacy Reasoning in Ambiguous Contexts (NeurIPS 2025 · similarity 0.89)
• Beyond Memorization: Violating Privacy via Inference with Large Language Models (ICLR 2024 · similarity 0.88)
• Empowering Users in Digital Privacy Management through Interactive LLM-Based Agents (ICLR 2025 · similarity 0.87)
• PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training (ICML 2026 · similarity 0.87)