Research map / Large language models / AI and society

Position: Contextual Integrity is Inadequately Applied to Language Models

ICML 2025 · Large language models › AI and society · approach: Critiques of LLM claims and judging · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing that LLM privacy research applying Contextual Integrity (CI) departs from the theory's four core tenets, systematizing nine prior works and flagging prompt-sensitivity and position-bias gaps in experimental practice.

Problem

Existing LLM privacy studies that invoke Contextual Integrity often reduce it to data-type protection, data minimization, compliance, or preference surveys, which can yield incorrect privacy conclusions and flawed designs.

Approach

Clarifies the four fundamental tenets of CI theory (privacy as appropriate flow, norm conformance, five-parameter flow description, and the CI heuristic for normative legitimacy) and uses them as a yardstick to audit nine prior CI-based LLM works, alongside a call for non-adversarial robustness checks.

Key results

• Of nine prior CI-based LLM works surveyed, all deviate from at least one CI tenet; none fully satisfy the CI heuristic tenet (Table summary).
• None of the surveyed prior works account for paraphrasing prompt sensitivity or position bias in their LLM evaluations.
• Only CI-Bench is marked as aligned with the descriptive framework tenets (privacy as flow, five parameters) among the surveyed works; all nine deviate on the CI heuristic tenet.

Surprising findings

• None of the nine surveyed CI-based LLM works mention or apply the CI heuristic, despite claiming to use CI.
• Several works conflate regulatory compliance (e.g., HIPAA) and crowdsourced preferences with privacy norms.

Limitations

• Conceptual argument with no new experiments; evaluation of prior work is qualitative and based on the authors' reading.
• Systematization covers only nine works, selected by the authors.

Experimental setup

Baselines: ConfAIde GoldCoin PrivacyChecklist Airgap PrivacyLens CI-Bench

Builds on: Contextual Integrity (Nissenbaum)

Closest papers by meaning

• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.93)
• It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMs (NeurIPS 2026 · similarity 0.89)
• Contextual Integrity in LLMs via Reasoning and Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
• Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning (ICML 2025 · similarity 0.88)
• Beyond Memorization: Violating Privacy via Inference with Large Language Models (ICLR 2024 · similarity 0.88)
• Privacy Auditing of Large Language Models (ICLR 2025 · similarity 0.87)