Research map / Federated learning and privacy / Differential privacy
COVERT: Privacy-Preserving Covariant Obfuscation for VLMaaS via Exact Reparameterization and Tailored Tuning
Wenqiang Ruan, ZiRui Huang, Yu Lin, Qizhi Zhang, Yunlong Mao, Quanwei Cai, Jue Hong, Sheng Zhong, Ye Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add tailored utility-aware tuning of first ViT block attention (ablation) | COVERT obfuscation · ViT hidden-state inversion (adaptive attack), layer 1 | SSIM: 0.3404 -> 0.1550 | Improved |
| add tailored utility-aware tuning of first ViT block attention (ablation) | COVERT obfuscation · ViT hidden-state inversion (adaptive attack), layer 1 | MSE: 3275.28 -> 4577.28 | Improved |
| scale model from 7B to 32B | COVERT · Five VLM benchmarks (avg of 5 datasets) | average accuracy drop: 2.58 -> 1.83 | Improved |
| scale model from 7B to 72B | COVERT · Five VLM benchmarks (avg of 5 datasets) | average accuracy drop: 2.58 -> 2.28 | Improved |
| add 8-bit quantization of obfuscated model | COVERT (Qwen2.5-VL-7B) · Five VLM benchmarks (avg of 5 datasets) | average accuracy: 68.63 -> 68.40 | No clear effect |
| apply COVERT to Fara-7B and Mimo-7B variants | COVERT · Five VLM benchmarks (avg of 5 datasets) | average accuracy drop: 1.53 and 2.81 drop | No clear effect |
| replace DP-Forward (eps=128) with COVERT | DP-Forward · Dataset 1 (Qwen2.5-VL-7B) | accuracy: 0.07 -> 79.79 | Improved |
| replace DP-Forward (eps=128) with COVERT | DP-Forward · Dataset 4 (Qwen2.5-VL-7B) | accuracy: 13.76 -> 83.87 | Improved |
| add COVERT obfuscation to serving Qwen2.5-VL-32B on vLLM | Plaintext inference · Online inference, Qwen2.5-VL-32B vLLM | throughput: 25.17 -> 23.76 | Hurt |
| add COVERT obfuscation to serving Qwen2.5-VL-32B on vLLM | Plaintext inference · Online inference, Qwen2.5-VL-32B vLLM | TPOT: 26.73 -> 29.26 | Hurt |
| add 64-bin quantization and minimal patch expansion (ablation) | COVERT communication · Communication expansion | expansion ratio: 7.68x -> 1.30x | Improved |
| send obfuscated image as JPG instead of PNG (ablation) | COVERT communication · Communication expansion | expansion ratio: 7.68x -> 32.15x | Hurt |
Closest papers by meaning
- VERA-V: Variational Inference Framework for Jailbreaking Vision-Language Models (ICML 2026 · similarity 0.87)
- Steal the Patch Size: Adversarially Manipulate Vision Language Models (ICML 2026 · similarity 0.86)
- Backdooring Vision-Language Models with Out-Of-Distribution Data (ICLR 2025 · similarity 0.86)
- FusionAudit: Pathway-Conditioned Robustness Auditing for Native Multimodal Models (NeurIPS 2026 · similarity 0.86)
- Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset (ICLR 2025 · similarity 0.85)
- Private Attribute Inference from Images with Vision-Language Models (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
