Research map / Detection and segmentation / Open-set and few-shot detection
MOS: Modeling Object-Scene Associations in Generalized Category Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MOS, a dual-branch GCD framework that uses object-only and original images plus an MLP scene-awareness module to exploit scene information as a prior, improving fine-grained generalized category discovery over SimGCD.
Problem
Generalized category discovery (GCD) methods typically treat scene information as noise, and the paper argues scene context is a useful prior for inferring novel classes, but is misinterpreted due to an ambiguity challenge between base/novel objects and scenes.
Approach
Decouple objects from scenes using zero-shot saliency segmentation (IS-Net), then run a shared ViT backbone on both object and original images, treating the original-image features as scene context, and fuse them with a lightweight MLP scene-awareness module on top of SimGCD.
Key results
- MOS reaches 69.6 All accuracy on CUB, 64.6 on Stanford Cars and 61.1 on FGVC-Aircraft, averaging 65.1 versus SimGCD's 56.1 (about 9 points).
- On Oxford-IIIT Pet, MOS achieves 93.2 All accuracy versus SimGCD's 88.8.
- Ablation on CUB shows the full MOS framework at 69.57 All accuracy versus 61.50 for the baseline without object branch or scene module.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add object-scene association modeling with the MOS framework | SimGCD · CUB All | ACC: 69.4 -> 69.6 | Helped |
| add object-scene association modeling with the MOS framework | SimGCD · Stanford Cars All | ACC: 59.0 -> 64.6 | Helped |
| add object-scene association modeling with the MOS framework | SimGCD · FGVC-Aircraft All | ACC: 59.3 -> 61.1 | Helped |
| add object-scene association modeling with the MOS framework | SimGCD · Oxford-IIIT Pet All | ACC: 91.8 -> 93.2 | Helped |
| use object images for traditional training (ablation) | SimGCD on CUB · CUB All | ACC: 61.50 -> 63.06 | Helped |
| add MOS components except the scene-awareness module (ablation) | object-image training on CUB · CUB All | ACC: 63.06 -> 67.86 | Helped |
| add the scene-awareness module (ablation) | MOS without SA Module on CUB · CUB All | ACC: 67.86 -> 69.57 | Helped |
Surprising findings
- Removing scene information from images improves GCD performance mainly when base-novel relationships between object and scene conflict, suggesting scene information is harmful only under ambiguity.
Limitations
- Struggles to extract effective object information at extremely low resolution such as CIFAR-10/100.
- Performance depends on zero-shot saliency segmentation quality, which produces poor cases (acknowledged via visualization).
Experimental setup
Datasets: CUB Stanford Cars FGVC-Aircraft Oxford-IIIT Pet ImageNet-100
Benchmarks: CUB Stanford Cars FGVC-Aircraft Oxford-IIIT Pet ImageNet-100
Metrics: clustering accuracy (ACC)
Baselines: k-means++ RankStats+ UNO+ GCD XCon ORCA DCCL GPC PIM SimGCD PromptCAL InfoSieve
Builds on: SimGCD DINO IS-Net Hungarian matching
Closest papers by meaning
- PartCo: Part-Level Correspondence Priors Enhance Category Discovery (ICML 2026 · similarity 0.84)
- Adaptive Part Learning for Fine-Grained Generalized Category Discovery: A Plug-and-Play Enhancement (CVPR 2025 · similarity 0.83)
- Multi-Part Object Representations via Graph Structures and Co-Part Discovery (NeurIPS 2026 · similarity 0.82)
- CQ-DINO: Mitigating Gradient Dilution via Category Queries for Vast Vocabulary Object Detection (NeurIPS 2025 · similarity 0.81)
- CAKE: Category Aware Knowledge Extraction for Open-Vocabulary Object Detection (AAAI 2025 · similarity 0.81)
- SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.