Research map / Multi-agent systems and games / Human-AI and goal-conditioned
PromptPilot: Game-Theoretic Multi-Agent Prompt Optimization for Segment Anything
Guangze Shi, Yingjie Mi, Jia Shen, Feixue Shao, Jiarui Cao, Yexin Lai, Xueyu Liu, Rui Wang, Yongfei Wu, Mingqiang Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add manager agent with SAM DSC feedback to feature-only agent (ablation) | single feature agent · 4 medical datasets avg | mIoU: 34.0 -> 52.2 | Improved |
| add manager agent with SAM DSC feedback to physical-only agent (ablation) | single physical agent · 4 medical datasets avg | mIoU: 34.5 -> 53.0 | Improved |
| combine feature and physical agents under the manager (ablation) | DSC-manager variant · 4 medical datasets avg | mIoU: 52.2 -> 61.3 | Improved |
| add LOO marginal contribution with EMA to the manager reward (ablation) | PromptPilot · 4 medical datasets avg | mIoU: 61.3 -> 62.9 | Improved |
| use feature objective only versus physical objective only (single agent) (ablation) | single-agent prompt optimization · 4 medical datasets avg | mIoU: 34.0 -> 34.5 | No clear effect |
| use one monolithic DQN agent for both objectives instead of separate agents (ablation) | single-agent prompt optimization · 4 medical datasets avg | mIoU: 34.2 | No clear effect |
| replace Matcher heuristic prompt selection with PromptPilot | Matcher · COCO | mIoU: +8.7 | Improved |
| replace single-agent PPO with multi-agent PromptPilot | FM-PPO · VOC and COCO | mIoU: +8.5 and +16.7 | Improved |
| replace FM-PPO with PromptPilot on feature-matching initial prompts | FM-PPO · 4 medical datasets avg | mIoU: 57.0 -> 62.9 | Improved |
| apply PromptPilot to coarse UNet pseudo-label prompts | Coarse Segmentation baseline · 4 medical datasets avg | mIoU: 25.0 -> 61.2 | Improved |
| replace PPO with PromptPilot on coarse segmentation prompts | CS-PPO · 4 medical datasets avg | mIoU: 28.6 -> 61.2 | Improved |
| add PromptPilot over Matcher on DAVIS 2017 | Matcher · DAVIS 2017 | region similarity J: 65.1 -> 69.3 | Improved |
Closest papers by meaning
- Plug-and-Play PPO: An Adaptive Point Prompt Optimizer Making SAM Greater (CVPR 2025 · similarity 0.87)
- SePO: Self-Evolving Prompt Agent for System Prompt Optimization (NeurIPS 2026 · similarity 0.84)
- LENS: Learning to Segment Anything with Unified Reinforced Reasoning (AAAI 2026 · similarity 0.84)
- Rethinking Interactive Image Segmentation with Low Latency High Quality and Diverse Prompts (CVPR 2024 · similarity 0.84)
- Prompting Diffusion Models for Zero-Shot Instance Segmentation (NeurIPS 2026 · similarity 0.83)
- AlignSAM: Aligning Segment Anything Model to Open Context via Reinforcement Learning (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
