Research map / Large language models / Language, safety and interpretability

Privacy Reasoning in Ambiguous Contexts

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Faithfulness of LLM explanations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that LLM privacy judgments on information-sharing scenarios are hurt by context ambiguity, and that reasoning-guided context disambiguation (Camber) improves precision/recall and reduces prompt sensitivity on augmented PrivacyLens and ConfAIde datasets.

Problem

LLMs making appropriateness judgments about information sharing perform poorly and are prompt-sensitive, partly because scenarios are underspecified (context ambiguity), which personal agents may face in practice.

Approach

Elicit model reasoning on yes/no privacy judgments, code the implicit assumptions into nine privacy codes, and generate label-dependent context expansions that address those assumptions, simulating user clarifications; compare against label-independent and label-dependent field-level expansions.

Key results

• Reasoning-guided expansion with the consent code raises PrivacyLens+ precision from 86.5% to 99.8% (+13.3) and recall from 69.0% to 91.3% (+22.3) for Gemini 2.5 Pro.
• Reasoning-guided expansion improves F1 by 8.9-18.5% on PrivacyLens+ and 0.9-8.2% on ConfAIde+.
• Prompt sensitivity on PrivacyLens+ drops from 9.8% F1 range on original examples to between 3.9% and 8.7% across codes.

Measured effects reported in the paper

Surprising findings

• Expanding context based on the code named in the model's reasoning does not outperform expanding with other codes on a per-example basis, suggesting stated reasoning is an imperfect guide to the underlying inference.
• Label-independent context (neutral added information) provides little or no benefit over no expansion.

Limitations

• Expansions are synthetic and may not be representative of real-world user clarifications.
• The framework does not provide a mechanism for obtaining user clarifications in practice.
• Codes derived from LLM reasoning may be shaped by the datasets used and may not generalize to agent interactions.

Experimental setup

Datasets: PrivacyLens ConfAIde

Benchmarks: PrivacyLens+ ConfAIde+

Metrics: precision recall F1 prompt sensitivity entropy

Baselines: No expansion Label-independent expansion Label-dependent expansion

Builds on: contextual integrity PrivacyLens ConfAIde

Closest papers by meaning

• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.90)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action (NeurIPS 2024 · similarity 0.89)
• Contextual Integrity in LLMs via Reasoning and Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
• PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training (ICML 2026 · similarity 0.87)
• Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark (ICLR 2026 · similarity 0.87)
• Position: Contextual Integrity is Inadequately Applied to Language Models (ICML 2025 · similarity 0.86)