Research map / Detection and segmentation / Open-set and few-shot detection
Visual Prompting via Image Inpainting
Amir Bar, Yossi Gandelsman, Trevor Darrell, Amir Globerson, Alexei A Efros
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain the inpainting model on Computer Vision Figures rather than ImageNet alone (ablation) | Visual prompting inpainting models · Downstream image-to-image tasks | task-specific metric | Improved |
| Add ImageNet images to Figures pretraining (ablation) | MAE-VQGAN · Pascal-5i Foreground Segmentation | mIOU | Improved |
| Increase the number of input-output examples in the visual prompt (ablation) | MAE-VQGAN (Figures) · Pascal-5i Foreground Segmentation | mIOU | Improved |
| Use a vertical rather than horizontal prompt layout (ablation) | MAE-VQGAN (Figures) · Pascal-5i Foreground Segmentation | mIOU: 27.17 -> 31.57 | Improved |
| Use black/white rather than purple/yellow mask colors (ablation) | MAE-VQGAN (Figures) · Pascal-5i Foreground Segmentation, Vertical layout | mIOU: 28.47 -> 31.57 | Improved |
| Ensemble different prompt layouts (ablation) | MAE-VQGAN (Figures) · Synthetic study Color | color-aware mIOU: 39.97 -> 44.14 | Improved |
| Ensemble different prompt layouts (ablation) | MAE-VQGAN (Figures) · Synthetic study Shape | color-aware mIOU: 46.54 -> 60.42 | Improved |
| Ensemble different prompt layouts (ablation) | MAE-VQGAN (Figures) · Synthetic study Size | color-aware mIOU: 42.01 -> 49.42 | Improved |
| Use MAE-VQGAN instead of the strongest reported alternative | Visual prompting · Pascal-5i Foreground Segmentation | mIOU: 19.57 -> 27.17 | Improved |
| Use MAE-VQGAN instead of the strongest reported alternative | Visual prompting · Pascal VOC 2012 Single Object Detection | mIOU: 5.39 -> 25.00 | Improved |
| Use MAE-VQGAN instead of the best reported model for MSE | Visual prompting · ImageNet validation Colorization | MSE: 0.43 -> 0.67 | Hurt |
| Use MAE-VQGAN instead of the strongest reported alternative | Visual prompting · ImageNet validation Colorization | LPIPS: 0.56 -> 0.40 | Improved |
Closest papers by meaning
- Visual In-Context Prompting (CVPR 2024 · similarity 0.90)
- AutoVP: An Automated Visual Prompting Framework and Benchmark (ICLR 2024 · similarity 0.89)
- Brush2Prompt: Contextual Prompt Generator for Object Inpainting (CVPR 2024 · similarity 0.89)
- SA²VP: Spatially Aligned-and-Adapted Visual Prompt (AAAI 2024 · similarity 0.89)
- LaViP: Language-Grounded Visual Prompting (AAAI 2024 · similarity 0.89)
- Enhancing Visual Prompting through Expanded Transformation Space and Overfitting Mitigation (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
