Research map / Multi-agent systems and games / LLM agents

PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PrivAct, which trains each agent in a generator-verifier-refiner LLM system with DPO on tree-propagated preference data and a leakage-conditioned asymmetric reward, to internalize contextual privacy and reduce leakage on PrivacyLens.

Problem

LLM agents acting on sensitive, context-dependent user data leak private information in their actions; existing defenses rely on inference-time prompts or gatekeeper agents that are brittle and expand the attack surface via reasoning traces.

Approach

Propagate final-response rewards upward through a branching multi-agent tree (average of descendant values) to create per-agent DPO preference pairs, and shape the reward asymmetrically so that helpfulness only counts when leakage is zero (LC-ARS), internalizing privacy into each agent's generation policy.

Key results

• On Mistral-7B, PrivAct (b1=0, b2=0.3) reduces average privacy leakage to 11.4% versus 23.7% for AIFC, while AIFC has 0.63% higher helpfulness (86.93% vs 86.30%).
• Zero-shot transfer from PrivacyLens-trained refiner/verifier improves ConfAIde tier 3 and tier 4 metrics across Llama-8B, Llama-3B, Mistral-7B and Qwen-4B backbones.
• Across MAS topologies, PrivAct lowers average privacy leak versus vanilla (e.g., 4.242 vs 8.788 on one topology) while keeping helpfulness comparable or higher.

Surprising findings

• Training for privacy-compliant acting also improved probing-based privacy evaluation (ConfAIde) without direct supervision on probing tasks.
• AIFC, an agentic privacy gatekeeper, achieved higher helpfulness than PrivAct on Mistral-7B but with substantially higher leakage.

Limitations

• Relies on fine-tuning, limiting applicability to closed-source models (stated in impact statement).
• Leakage and helpfulness judged by an LLM judge (Mistral-7B-Instruct-v0.2) rather than humans.
• Requires retraining under multiple (b1,b2) configurations to trace the tradeoff frontier.

Experimental setup

Datasets: PrivacyLens ConfAIde

Benchmarks: PrivacyLens ConfAIde

Metrics: privacy leak avg leak@K helpfulness avg helpfulness bin ConfAIde tier 3 and tier 4 error rates

Baselines: Vanilla LM Prompt-based Privacy Enforcement (PPE) Agent-based Information Flow Control (AIFC) 1-2-3 Check

Builds on: direct preference optimization PrivacyLens 1-2-3 Check Sirius MALT

Closest papers by meaning

• Contextualized Privacy Defense for LLM Agents (ICML 2026 · similarity 0.90)
• It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMs (NeurIPS 2026 · similarity 0.89)
• Contextual Integrity in LLMs via Reasoning and Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
• MINIM: Privacy-Aware Minimal View for Agents via Trusted Local Sanitization (ICML 2026 · similarity 0.88)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.87)
• Privacy Reasoning in Ambiguous Contexts (NeurIPS 2025 · similarity 0.87)