Research map / Adversarial robustness and security / Attacks on text and graphs
CosalPure: Learning Concept from Group Images for Robust Co-Saliency Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CosalPure, a diffusion-based purification framework that learns a shared concept token from a group of images via textual inversion and uses it to guide diffusion purification, restoring co-salient object detection robustness against Jadena attacks.
Problem
Co-salient object detectors are highly vulnerable to adversarial perturbations (exposure and noise attacks) applied to a subset of images in a group, and existing diffusion purification (DiffPure) ignores object semantics, introducing artifacts that hurt CoSOD.
Approach
Learn a concept token for the co-salient objects from the whole (partly adversarial) image group via textual inversion on a pre-trained T2I diffusion model, then embed this concept as an extra condition in the diffusion purification (reverse process) of each image, so purification preserves object semantics.
Key results
- On Cosal2015 with GICD, CosalPure raises success rate from 0.3493 (source-only) and 0.4595 (DiffPure) to 0.5602.
- On iCoseg with GICD, success rate improves to 0.5396 versus 0.4447 for DiffPure and 0.4665 for DDA.
- Under motion blur on Cosal2015, CosalPure achieves SR 0.4575 versus 0.3915 for source-only and 0.3146 for DiffPure.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply COSALPURE rather than DDA | GICD · Cosal2015; iCoseg; CoSOD3k; CoCA | SR: 0.4565 -> 0.5602; 0.4665 -> 0.5396; 0.3838 -> 0.4659; 0.2054 -> 0.2409 | Helped |
| Apply COSALPURE rather than DDA | GCAGC · Cosal2015; iCoseg; CoSOD3k; CoCA | SR: 0.5955 -> 0.5975; 0.6982 -> 0.7060; 0.5337 -> 0.5946; 0.2671 -> 0.3057 | Helped |
| Apply COSALPURE rather than DDA | PoolNet · Cosal2015; iCoseg; CoSOD3k; CoCA | SR: 0.6233 -> 0.6908; 0.6578 -> 0.7278; 0.5105 -> 0.5859; 0.2416 -> 0.2633 | Helped |
| Add concept-guided purification rather than omit concept inversion (ablation) | GICD · Cosal2015; CoSOD3k | SR: 0.5186 -> 0.5602; 0.4424 -> 0.4659 | Helped |
| Add the learned concept rather than use None during purification (ablation) | GICD · Cosal2015; CoSOD3k | SR: 0.5225 -> 0.5602; 0.4297 -> 0.4659 | Helped |
| Add concept-guided purification rather than omit concept inversion (ablation) | GCAGC · Cosal2015; CoSOD3k | SR: 0.5528 -> 0.5975; 0.5753 -> 0.5946 | Helped |
| Add the learned concept rather than use None during purification (ablation) | GCAGC · Cosal2015; CoSOD3k | SR: 0.5334 -> 0.5975; 0.5488 -> 0.5946 | Helped |
| Add concept-guided purification rather than omit concept inversion (ablation) | PoolNet · Cosal2015; CoSOD3k | SR: 0.6843 -> 0.6908; 0.5747 -> 0.5859 | Helped |
| Add the learned concept rather than use None during purification (ablation) | PoolNet · Cosal2015; CoSOD3k | SR: 0.6774 -> 0.6908; 0.5711 -> 0.5859 | Helped |
| Apply COSALPURE rather than DiffPure | CoSOD methods · Cosal2015 motion blur | SR: 0.3146 -> 0.4575 | Helped |
Surprising findings
- DiffPure can lower CoSOD success rate below the source-only baseline for some detector/dataset pairs (e.g., CoCA with GICD: 0.1706 vs 0.1837 source-only), and under motion blur DiffPure drops to 0.3146 from 0.3915.
- CosalPure sometimes has a lower clean-image success rate than DDA or source-only, trading clean accuracy for adversarial recovery.
Limitations
- Clean-image success rate can drop relative to source-only or DDA.
- Relies on the assumption that perturbed images do not corrupt concept learning, validated only empirically on a few examples.
- Evaluated only on Jadena attack and motion blur with three CoSOD detectors.
Experimental setup
Datasets: Cosal2015 iCoseg CoSOD3k CoCA ImageNet
Benchmarks: Cosal2015 iCoseg CoSOD3k CoCA
Metrics: success rate average precision F-measure MAE
Baselines: DiffPure DDA Source-Only
Builds on: DiffPure textual inversion DDPM Jadena
Closest papers by meaning
- Co-Salient Object Detection With Uncertainty-Aware Group Exchange-Masking (CVPR 2023 · similarity 0.92)
- Can You Spot the Chameleon? Adversarially Camouflaging Images From Co-Salient Object Detection (CVPR 2022 · similarity 0.91)
- Memory-Aided Contrastive Consensus Learning for Co-salient Object Detection (AAAI 2023 · similarity 0.91)
- CoADNet: Collaborative Aggregation-and-Distribution Networks for Co-Salient Object Detection (NeurIPS 2020 · similarity 0.91)
- Visual Consensus Prompting for Co-Salient Object Detection (CVPR 2025 · similarity 0.88)
- DiMSOD: A Diffusion-Based Framework for Multi-Modal Salient Object Detection (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.