Research map / Vision-language and multimodal / Multimodal LLMs
Steering Representations, Safeguarding Privacy: A Cross-Modal Privacy Protection Method for Generative AI
Jie Zhang, Chenxu Niu, Zhefeng Nan, Yangyan Xu, Jinta Weng
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a directional steering framework in representation space to enhance privacy protection in generative AI models across language and vision-language modalities.
Problem
Privacy protection in generative AI models that is independent of specific data and tasks, as models' privacy awareness raises new challenges.
Approach
Uses directional steering of internal representations to protect privacy, integrating with both language and vision-language models; the abstract is truncated so further specifics are not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add privacy steering vector (pv) | llama-2-13b-chat · Solove-Judge | accuracy: 0.4961 -> 0.6866 | Improved |
| add privacy steering vector (pv) | mistral-7b-0.1 · SALAD-Privacy | safe rate: 0.6317 -> 0.8234 | Improved |
| add privacy steering vector (pv) | llama-2-13b-chat · SALAD-Privacy | safe rate: 0.8265 -> 0.7489 | Hurt |
| add privacy steering vector (pv) | llama-2-7b-chat · Solove-Judge | accuracy: 0.6297 -> 0.6277 | No clear effect |
| add privacy steering vector (pv) | qwen-2.5-14b · MMLU | accuracy: 0.7760 -> 0.7758 | No clear effect |
| add privacy steering vector (pv) | mistral-7b-0.1 · LAMBADA | perplexity: 3.1818 -> 6.6412 | Hurt |
| use vector from LoRA fine-tuned model (ft pv) instead of own pv (ablation) | llama-2-7b · Solove-Judge | accuracy: 0.5452 -> 0.5717 | Improved |
| use vector from LoRA fine-tuned model (ft pv) instead of own pv (ablation) | llama-3-8b · Solove-Judge | accuracy: 0.5147 -> 0.5295 | Improved |
| use same-source base-model vector (ss pv) instead of own pv (ablation) | llama-2-7b-chat · Solove-Judge | accuracy: 0.6277 -> 0.6591 | Improved |
| use same-source base-model vector (ss pv) instead of own pv (ablation) | qwen-2.5-7b-it · Solove-Judge / Confaide 2a | accuracy / score: 0.8114 and -0.663 -> 0.8330 and -37.90 | Improved |
| transfer LLM steering vector to VLM (l2v pv) | llava-1.6-vicuna-7b · Ch3Ef | safe choices: 12/42 -> 22/42 | Improved |
| transfer LLM steering vector to VLM (l2v pv) | llava-1.6-vicuna-7b · MM-SafetyBench | safe ratio: 0.0863 -> 0.1654 | Improved |
Closest papers by meaning
- Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark (ICLR 2026 · similarity 0.88)
- MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models (ICML 2026 · similarity 0.88)
- PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training (ICML 2026 · similarity 0.87)
- Privacy-Preserving Instructions for Aligning Large Language Models (ICML 2024 · similarity 0.87)
- A Privacy-Preserving Framework for Generative Model-driven Synthetic Datasets (AAAI 2025 · similarity 0.87)
- PFGuard: A Generative Framework with Privacy and Fairness Safeguards (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
