Research map / Image generation / Text-to-image diffusion
Stable Diffusion Models are Secretly Good at Visual In-Context Learning
Trevine Oorloff, Vishwanath Sindagi, Wele Gedara Chaminda Bandara, Ali Shafahi, Amin Ghiasi, Charan Prakash, Reza Ardekani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use implicitly-weighted prompt ensembling (IWPE) with 5 prompts instead of 1 prompt (ablation) | SD-VICL · Pascal-5i foreground segmentation | mIoU: 43.92 -> 55.49 | Improved |
| use IWPE instead of SegGPT feature ensembling (5 prompts) (ablation) | SD-VICL · Cityscapes semantic segmentation | mIoU: 22.97 -> 27.01 | Improved |
| use IWPE instead of feature ensembling on simpler single-class tasks (ablation) | SD-VICL · tasks other than semantic segmentation | task metrics | No clear effect |
| use query from query image and key from prompt source (proposed attention) instead of alternatives (ablation) | SD-VICL · Pascal-5i foreground segmentation | mIoU: 23.68 -> 55.49 | Improved |
| add AdaIN color alignment (ablation) | SD-VICL · Pascal-5i foreground segmentation | mIoU: 51.55 -> 55.49 | Improved |
| modify self-attention layers at all three resolutions instead of one (ablation) | SD-VICL · Pascal-5i foreground segmentation | mIoU: 50.33 -> 55.49 | Improved |
| use five prompts with 30 denoising steps instead of one prompt with 70 steps (ablation) | SD-VICL · Pascal-5i foreground segmentation | mIoU / inference time: 38.4 s (1 prompt, 70 steps) vs 35.9 s (5 prompts, 30 steps) | Improved |
| increase number of prompts from 1 to 5 (ablation) | SD-VICL · inference speed | inference speed | Hurt |
| add text guidance to IMProv for keypoint detection | IMProv · DeepFashion keypoint detection | MSE: 31.30 -> 44.67 | Hurt |
| use contrast strength above 1.0 and enable swap-guidance (ablation) | SD-VICL · Pascal-5i foreground segmentation | mIoU | Improved |
Closest papers by meaning
- Explore In-Context Segmentation via Latent Diffusion Models (AAAI 2025 · similarity 0.89)
- Visual In-Context Prompting (CVPR 2024 · similarity 0.88)
- InvSeg: Test-Time Prompt Inversion for Semantic Segmentation (AAAI 2025 · similarity 0.88)
- Hugging Visual Prompt and Segmentation Tokens: Consistency Learning for Fine-Grained Visual Understanding in MLLMs (CVPR 2026 · similarity 0.88)
- In-Context Learning Can Help Vision Language Models Overcome Training Prior (NeurIPS 2026 · similarity 0.88)
- I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
