Research map / Image generation / Text-to-image diffusion
No Caption, No Problem: Caption-Free Membership Inference via Model-Fitted Embeddings
Joonsung Jeon, Woo Jae Kim, Suhyeon Ha, Sooel Son, Sung-eui Yoon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace ground-truth captions with VLM-generated captions | CLiD · MIA Pokemon | ASR (%): 96.52 -> 77.55 | Hurt |
| condition on model-fitted embedding instead of VLM captions | MoFit vs CLiD · MIA Pokemon | ASR (%): 77.55 -> 94.48 | Improved |
| condition on model-fitted embedding instead of ground-truth captions | MoFit vs CLiD (GT) · MIA MS-COCO | ASR (%): 86.50 -> 88.00 | Improved |
| MoFit versus CLiD with GT captions | MoFit · MIA Pokemon | ASR (%): 96.52 -> 94.48 | Hurt |
| extract embedding from optimized surrogate x0+delta* instead of raw query image (ablation) | MoFit · MIA MS-COCO | TPR@1%FPR: 31.00 -> 47.00 | Improved |
| extract embedding from surrogate instead of random-noise-perturbed image (ablation) | MoFit · MIA Pokemon | TPR@1%FPR: 10.34 -> 50.48 | Improved |
| use adversarial delta_MAX instead of surrogate optimization (ablation) | MoFit · MIA Pokemon | ASR (%): 75.63 -> 75.87 | No clear effect |
| fine-tune target with LoRA instead of full U-Net | MIA attacks · MIA SD v1.4 Pokemon LoRA | ASR (%): 94.48 -> 58.50 | Hurt |
| fine-tune with Gaussian blur or JPEG augmentation | MoFit · MIA MS-COCO | ASR (%): 88.00 -> 88.70 (blur) / 82.80 (JPEG) | Mixed |
| use MoFit on SD v1.5 relative to VLM-captioned CLiD | MoFit · MIA SD v1.5 | TPR@1%FPR: 4.18 -> 41.30 | Improved |
| use MoFit versus PFAMI on SD v1.5 AUC | MoFit · MIA SD v1.5 | AUC: 78.15 -> 71.03 | Hurt |
Closest papers by meaning
- Variance-Based Membership Inference Attacks Against Large-Scale Image Captioning Models (CVPR 2025 · similarity 0.89)
- Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy (NeurIPS 2024 · similarity 0.88)
- Get What You Want, Not What You Don't: Image Content Suppression for Text-to-Image Diffusion Models (ICLR 2024 · similarity 0.86)
- Diffusion Bridge: Leveraging Diffusion Model to Reduce the Modality Gap Between Text and Vision for Zero-Shot Image Captioning (CVPR 2025 · similarity 0.86)
- A Comprehensive Study of Decoder-Only LLMs for Text-to-Image Generation (CVPR 2025 · similarity 0.86)
- On improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
