Research map / Detection and segmentation / Open-set and few-shot detection
Visual Consensus Prompting for Co-Salient Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Visual Consensus Prompting (VCP), a parameter-efficient prompt-tuning framework that freezes a SegFormer backbone and generates consensus-based visual prompts for co-salient object detection.
Problem
Co-salient object detection methods use a three-stage architecture with full fine-tuning of the foundation model, which is parameter-inefficient and gives consensus extraction no timely guidance to the encoder.
Approach
Embeds intra-group consensus extraction into visual prompts: a Consensus Prompt Generator uses learnable saliency seeds and top-k consensus seeds to produce consensus prompts, and a Consensus Prompt Disperser fuses them with embedding and FFT-based handcrafted prompts to tune frozen Transformer layers at different depths.
Key results
- On CoCA, VCP reaches Sm 0.819 and Fmax 0.752, a 5.6% Sm and 6.8% Fm gain over the best prior method, outperforming 13 full fine-tuning and prompt-based methods.
- VCP uses about 4.94M tunable parameters, versus 392.85M for CADC and 280.36M for GCoNet on the same comparison table.
- Replacing consensus prompts with Adaptformer, VPT-Deep, or EVP prompts yields CoCA Sm of 0.695-0.704, well below VCP's 0.774 in the ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add consensus prompts to the prediction-head baseline (ablation) | VCP · CoCA | Sm: 0.635 -> 0.763 | Helped |
| add handcrafted prompts (ablation) | VCP · CoCA | Sm: 0.659 -> 0.704 | Helped |
| combine handcrafted and consensus prompts by concatenation rather than addition (ablation) | VCP · CoCA | Sm: 0.755 -> 0.774 | Helped |
| remove classification loss (ablation) | VCP · CoCA | Sm: 0.761 -> 0.774 | Hurt |
| remove multiscale prediction supervision (ablation) | VCP · CoCA | Sm: 0.742 -> 0.774 | Hurt |
| replace the custom prediction head with the Segformer head (ablation) | VCP · CoCA | Sm: 0.747 -> 0.774 | Hurt |
| use shared MLP layers for prompt dispersal (ablation) | VCP · CoCA | Sm: 0.768 -> 0.774 | Hurt |
| use unshared MLP layers for prompt dispersal (ablation) | VCP · CoCA | Sm: 0.763 -> 0.774 | Hurt |
| fine-tune the full model instead of using parameter-efficient tuning (ablation) | VCP · CoCA | Sm: 0.740 -> 0.774 | Hurt |
| replace VCP prompts with Adaptformer prompts (ablation) | VCP · CoCA | Sm: 0.695 -> 0.774 | Hurt |
| replace VCP prompts with VPT-Deep prompts (ablation) | VCP · CoCA | Sm: 0.700 -> 0.774 | Hurt |
| replace VCP prompts with EVP prompts (ablation) | VCP · CoCA | Sm: 0.704 -> 0.774 | Hurt |
Surprising findings
- Retrained EVP, a prompt-tuning SOD method, performs poorly on CoSOD (CoCA Sm 0.686 with 3.7M tunable parameters).
- The VCP prompt-tuned model (CoCA Sm 0.774) outperforms its own full fine-tuning variant (0.740) in the ablation.
Limitations
- Evaluated only at 288x288 input resolution on three CoSOD benchmarks.
- Relies on a single backbone (SegFormer) and only three benchmark datasets.
Experimental setup
Datasets: DUT-class COCO-9k COCO-SEG
Benchmarks: CoCA CoSOD3k CoSal2015
Metrics: MAE Em Emax Fm Fmax Sm
Baselines: SCED MCCL GCoNet+ GEM CADC++ CoPR DMT UniTR CONDA EVP
Builds on: VPT EVP SegFormer AdaptFormer
Closest papers by meaning
- CosalPure: Learning Concept from Group Images for Robust Co-Saliency Detection (CVPR 2024 · similarity 0.88)
- CP-DETR: Concept Prompt Guide DETR Toward Stronger Universal Object Detection (AAAI 2025 · similarity 0.86)
- Just a Few Glances: Open-Set Visual Perception with Image Prompt Paradigm (AAAI 2025 · similarity 0.85)
- Visual Diversity and Region-aware Prompt Learning for Zero-shot HOI Detection (NeurIPS 2025 · similarity 0.85)
- SparseAlign: a Fully Sparse Framework for Cooperative Object Detection (CVPR 2025 · similarity 0.85)
- Spotting the Unseen: Reciprocal Consensus Network Guided by Visual Archetypes (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.