Research map / Detection and segmentation / Open-set and few-shot detection
Spider: A Unified Framework for Context-dependent Concept Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Spider, a single-parameter-set unified model for eight context-dependent concept segmentation tasks (natural scenes and medical lesions) that uses image-mask group prompts to generate dynamic concept filters.
Problem
Context-dependent (CD) concept segmentation tasks (camouflaged/salient/transparent objects, shadows, medical lesions) are typically solved by separate specialized models, limiting cross-domain generalisation; generalist models like SAM and SegGPT perform poorly on them.
Approach
A vanilla encoder-decoder with a concept filter: image-group prompts are encoded by a frozen encoder, and foreground/background mask-group prompts are refined via cross-attention into object-aware weights and context-aware bias that dynamically condition the prediction head, enabling 100% parameter sharing across eight tasks.
Key results
- Spider (ConvNeXt-L) reaches F_beta^w 0.8821 on DUTS and 0.7893 on COD10K, outperforming specialized models and SegGPT.
- Spider outperforms competitors by more than 30% on the TOS, CLI and BLS tasks, per the authors, with SegGPT scoring much lower (e.g., 0.3455 mDice on COVID-19 CT vs. 0.6956 for Spider-ConvNeXt-L).
- Continual learning fine-tuning <1% of parameters yields new-task gains with at most 5% degradation on old tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train jointly instead of separately (ablation) | Spider with ConvNeXt-B · DUTS | Fβω: 0.8593 -> 0.8732 | Helped |
| Add image-group prompts (ablation) | UNet baseline · DUTS | Fβω: 0.6253 -> 0.7843 | Helped |
| Add foreground mask-group prompts (ablation) | Spider concept filter · DUTS | Fβω: 0.7843 -> 0.8422 | Helped |
| Add background mask-group prompts (ablation) | Spider concept filter · DUTS | Fβω: 0.8422 -> 0.8732 | Helped |
| Replace concept filter with element-wise addition fusion (ablation) | Spider · DUTS | Fβω: 0.8732 -> 0.6534 | Hurt |
| Use balanced forward propagation and unified backpropagation (ablation) | Spider unified framework · DUTS | Fβω: 0.8608 -> 0.8732 | Helped |
| Use separate gradient updates for tasks instead of unified backpropagation (ablation) | Spider unified framework · DUTS | Fβω: 0.8732 -> 0.8422 | Hurt |
| Increase randomly selected inference prompts from 1 to 64 (ablation) | Spider · DUTS | Fβω: 0.7038 -> 0.8723 | Helped |
| Use clustered instead of random selection for 64 prompts (ablation) | Spider · DUTS | Fβω: 0.8723 -> 0.8732 | No clear effect |
| Use low-quality mask prompts by dilation or erosion (ablation) | Spider · DUTS | Fβω: 0.8732 -> 0.8718; 0.8732 -> 0.8699 | Hurt |
| Fine-tune on new tasks using less than 1% of parameters | Spider pretrained on T1-T4 · New tasks T5-T8 and old tasks T1-T4 | performance: less than 5% degradation on old tasks; + 63% (T5), + 59% (T6), + 63% (T7), and + 44% (T8) | Mixed |
Surprising findings
- SegGPT (ViT-L, trained on over 250,000 images) generalizes poorly across CD tasks, scoring e.g. 0.3874 F_beta^w on DUTS.
- Clustered group prompts during inference perform about the same as random selection, indicating robustness to prompt choice.
Limitations
- Performance on unseen tasks is only preliminary and the paper notes it improves with data scale and diversity.
- Reported gains on some tasks (e.g., transparent object segmentation, COVID-19) are modest or Spider does not lead on all metrics (e.g., ConvNeXt-L BER/MAE on TOS).
Experimental setup
Datasets: DUTS COD10K SBU Trans10K five polyp datasets COVID-19 CT BUSI ISIC2018
Benchmarks: DUTS COD10K SBU Trans10K five polyp datasets COVID-19 CT BUSI ISIC2018
Metrics: weighted F-measure S-measure BER MAE mDice mIoU
Baselines: SegGPT UniverSeg EVP MENet PGNet FEDER FSPNet SILT SARA EBLNet RFENet LDNet
Builds on: SegGPT EVP UNet SAM
Closest papers by meaning
- MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment (AAAI 2024 · similarity 0.84)
- Interactive Medical Image Analysis with Concept-based Similarity Reasoning (CVPR 2025 · similarity 0.83)
- BiomedCoOp: Learning to Prompt for Biomedical Vision-Language Models (CVPR 2025 · similarity 0.83)
- ContextShift: A Controlled Benchmark for Context Dependence in Object Detection (NeurIPS 2026 · similarity 0.83)
- ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.83)
- Prompt as Knowledge Bank: Boost Vision-language model via Structural Representation for zero-shot medical detection (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.