Research map / Image generation / Fast diffusion sampling
ObCLIP: Oblivious CLoud-Device Hybrid Image Generation with Privacy Preservation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ObCLIP, an oblivious cloud-device hybrid text-to-image scheme that sends candidate prompts differing only in sensitive attributes to the cloud, runs early denoising there, and finishes on-device, with cache and batch-reuse accelerations.
Problem
Text-to-image diffusion services leak sensitive attributes in user prompts to the cloud server, while existing private methods (MPC/HE) are too costly and DP perturbation degrades utility; hybrid cloud-device schemes like Hybrid SD still send prompt embeddings to the server.
Approach
Replace the real prompt with a set of candidate prompts that differ only in sensitive attributes (gender, age, ethnicity), let the cloud run the first k denoising steps on all candidates, and have the client select its own latent and finish denoising locally; accelerate server side by caching attention/block outputs across timesteps and reusing pivot-prompt attention maps across candidates.
Key results
- On the candidate prompt dataset with Realistic Vision v4.0 and N=6, ObCLIP with k=10 and reuse achieves FID 109.76 vs 113.39 for the cloud model, at 1.55s vs 1.12s latency (and ~4.4-7.6x faster than vanilla oblivious generation).
- On MS-COCO with SD-v1.4 and BK-SDM-small, ObCLIP (k=10, with cache) gets FID 15.73 vs 13.86 for SD-v1.4, with FLOPs reduced from 18.53T to 5.84T (server-side).
- On MJHQ with SDXL and Koala-700m, ObCLIP (k=10, with cache) achieves FID 30.79 vs 30.67 for SDXL, with server FLOPs 45.11T vs 159.35T.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ObCLIP (k=10) with cache and attention reuse instead of Realistic Vision v4.0 | Candidate prompt generation, two attributes (gender + age) · Candidate prompt dataset, 2-attribute (gender + age) | FID: 113.39 -> 109.76 | Helped |
| Use ObCLIP (k=10) with cache instead of BK-SDM-small | SD-v1.4 + BK-SDM-small · MS-COCO 2014, 30k prompts | FID: 18.30 -> 15.73 | Helped |
| Use ObCLIP (k=10) with cache instead of SDXL | SDXL + Koala-700m · MJHQ, 5k prompts | FLOPs (T): 159.35T -> $45.11^*$T | Helped |
| Add cache (ablation) | ObCLIP (k=10), SD-v1.4 + BK-SDM-small · MS-COCO 2014, 30k prompts | FID: 15.65 -> 15.73 | Hurt |
| Add cache (ablation) | ObCLIP (k=5), SD-v1.4 + BK-SDM-small · MS-COCO 2014, 30k prompts | FID: 16.55 -> 16.45 | Helped |
| Add cache (ablation) | ObCLIP (k=10), SDXL + Koala-700m · MJHQ, 5k prompts | FID: 31.92 -> 30.79 | Helped |
| Add cache (ablation) | ObCLIP (k=5), SDXL + Koala-700m · MJHQ, 5k prompts | FID: 32.42 -> 31.97 | Helped |
| Use a larger switch point k (ablation) | ObCLIP hybrid generation · MS-COCO and MJHQ | FID | Helped |
| Use a timestep shift of Δt=8 (ablation) | ObCLIP with LCM-SDXL and Koala-700m · MJHQ, 5k prompts | FID: 33.25 -> 33.92 | Helped |
Surprising findings
- Server-guided initial steps yield better FID than client-first allocation when server involvement is limited, and prompt swaps early in denoising are hard to correct later.
Limitations
- Computation grows sub-linearly with the number of candidate prompts, so server cost scales with N.
- Privacy is only in-distinguishability among candidates; the candidate set is exposed and the scheme is limited to text prompts.
- Assumes semi-honest server and secure channel; no man-in-the-middle protection.
Experimental setup
Datasets: MS-COCO 2014 MJHQ
Benchmarks: MS-COCO 2014 MJHQ candidate prompt dataset (synthetic)
Metrics: FID IS CLIP score FLOPs latency
Baselines: Hybrid SD HE-Diffusion Vanilla oblivious generation SD-v1.4 BK-SDM-small Realistic Vision v4.0 SDXL LCM-SDXL
Builds on: Hybrid SD T-Gate Faster Diffusion DPM-Solver
Closest papers by meaning
- PID: Prompt-Independent Data Protection Against Latent Diffusion Models (ICML 2024 · similarity 0.85)
- PCEvolve: Private Contrastive Evolution for Synthetic Dataset Generation via Few-Shot Private Data and Generative APIs (ICML 2025 · similarity 0.84)
- United We Stand: Accelerating Privacy-Preserving Neural Inference by Conjunctive Optimization with Interleaved Nexus (AAAI 2024 · similarity 0.84)
- All Rivers Run to the Sea: Private Learning with Asymmetric Flows (CVPR 2024 · similarity 0.84)
- Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Training (NeurIPS 2024 · similarity 0.83)
- Privacy-Preserving Low-Rank Adaptation Against Membership Inference Attacks for Latent Diffusion Models (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.