Research map / Detection and segmentation / Open-set and few-shot detection
Detecting Open World Objects via Partial Attribute Assignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PASS, which selects and optimizes a small set of class-agnostic attributes from an LLM-generated pool via curriculum-scheduled partial optimal transport on top of a frozen OWL-ViT, for few-shot open world object detection.
Problem
Open world object detection in few-shot, real-world settings requires detecting unknown objects without confusing them with background; prior approaches rely on probabilistic objectness models or multistage attribute selection that is costly and error-prone.
Approach
Models attribute selection as a Partial Optimal Transport problem between visual object embeddings and an attribute pool, where transported mass indicates attribute relevance, and selects attributes via a curriculum schedule trained end-to-end with a POT distance plus cross-entropy loss.
Key results
- On the B/16 backbone, PASS reaches overall Task 1 U-mAP 12.1 and K-mAP 14.3 versus FOMO's 6.4 and 11.5 (per Table 2 overall columns).
- With the L/14 backbone, PASS reaches overall Task 1 U-mAP 19.1... and overall Task 2 PK/CK mAP of 38.5/38.9 against FOMO's 27.4/26.1, per Table 2 overall columns.
- Ablation on Aquatic and Surgery shows removing the POT distance loss drops U-mAP substantially (e.g., Surgery U-mAP 16.6 to 3.0 for B/16 at 100-shot).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| minimize the POT distance loss (ablation) | PASS · Aquatic | U-mAP: 10.1 -> 21.7 | Helped |
| minimize the POT distance loss (ablation) | PASS · Aquatic | K-mAP: 18.2 -> 53.9 | Helped |
| minimize the POT distance loss (ablation) | PASS · Surgery | U-mAP: 8.0 -> 16.6 | Helped |
| minimize the POT distance loss (ablation) | PASS · Surgery | K-mAP: 14.1 -> 45.6 | Helped |
| select a subset of attributes (ablation) | PASS · Aquatic | U-mAP: 15.5 -> 21.7 | Helped |
| select a subset of attributes (ablation) | PASS · Aquatic | K-mAP: 34.1 -> 53.9 | Helped |
| select a subset of attributes (ablation) | PASS · Surgery | U-mAP: 15.2 -> 16.6 | Helped |
| select a subset of attributes (ablation) | PASS · Surgery | K-mAP: 43.6 -> 45.6 | Helped |
| reweight selected attributes with their ID scores (ablation) | PASS · Aquatic | U-mAP: 18.3 -> 21.7 | Helped |
| reweight selected attributes with their ID scores (ablation) | PASS · Aquatic | K-mAP: 42.2 -> 53.9 | Helped |
| reweight selected attributes with their ID scores (ablation) | PASS · Surgery | U-mAP: 13.4 -> 16.6 | Helped |
| reweight selected attributes with their ID scores (ablation) | PASS · Surgery | K-mAP: 40.0 -> 45.6 | Helped |
Surprising findings
- Relying solely on the classification loss without minimizing the POT distance leads to poor generalization of attributes to unknown objects.
Limitations
- Relies on a frozen OWL-ViT backbone and a fixed LLM-generated attribute pool from GPT-3.5.
- Evaluated only on few-shot real-world datasets; traditional OWOD benchmarks reported as saturated and not used.
Experimental setup
Datasets: Aquatic Aerial Game Medical Surgery Objects365 Visual Genome
Benchmarks: Real-World Object Detection Benchmark (Aquatic, Aerial, Game, Medical, Surgery)
Metrics: U-mAP K-mAP PK-mAP CK-mAP
Baselines: FOMO BASE-ZS BASE-ZS+GT BASE-ZS+IN BASE-ZS+LLM BASE-FS
Builds on: OWL-ViT FOMO Partial Optimal Transport Sinkhorn optimal transport curriculum learning
Closest papers by meaning
- OW-VAP: Visual Attribute Parsing for Open World Object Detection (ICML 2025 · similarity 0.91)
- OW-OVD: Unified Open World and Open Vocabulary Object Detection (CVPR 2025 · similarity 0.91)
- Open-World Objectness Modeling Unifies Novel Object Detection (CVPR 2025 · similarity 0.91)
- Exploring Orthogonality in Open World Object Detection (CVPR 2024 · similarity 0.90)
- UMB: Understanding Model Behavior for Open-World Object Detection (NeurIPS 2024 · similarity 0.90)
- Looking Beyond the Known: Towards a Data Discovery Guided Open-World Object Detection (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.