Research map / Label-efficient and robust learning / Recognition and long-tail
From Coarse to Fine-Grained Open-Set Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Curates iNat2021-OSR open-set splits at 1-7 taxonomic hops to study fine-grained open-set recognition, and proposes hierarchy-adversarial learning via gradient reversal to improve fine-grained OSR.
Problem
Open-set recognition is usually studied with coarse semantic shifts, while real-world applications often involve fine-grained, visually similar novel categories; the role of label granularity, semantic similarity, and hierarchical representations in OSR is underexplored.
Approach
Builds taxonomy-hop-based open-set splits of iNat2021 and proposes hierarchy-adversarial learning, which uses a gradient reversal layer to maximize cross-entropy on coarser taxonomic levels while minimizing it on the finest level, discouraging hierarchical structure in representations.
Key results
- Hierarchy-adversarial learning (alpha=0.25) improves 1-hop fine-grained OSR MLS AUROC for Aves from 70.1 (cross-entropy) to 72.7 and AUPR from 65.5 to 70.2 in 5-model ensembles.
- MLS performs best for fine-grained OSR up to 6-hop, while KL-disagreement is best for coarse-grained OSR.
- Training on the finest granularity improves both open-set and closed-set performance even for coarse open-set splits.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add hierarchy-adversarial training (ablation) | ResNet-50 on Aves · Aves 1-hop | AUROC: 70.1 -> 72.7 | Helped |
| add hierarchy-adversarial training (ablation) | ResNet-50 on Aves · Aves 1-hop | AUPR: 65.5 -> 70.2 | Helped |
| add hierarchy-adversarial training (ablation) | ResNet-50 on Insecta · Insecta 1-hop | AUROC: 78.0 -> 79.5 | Helped |
| add hierarchy-adversarial training (ablation) | ResNet-50 on Insecta · Insecta 1-hop | AUPR: 76.4 -> 77.9 | Helped |
| add hierarchy-adversarial training (ablation) | ResNet-50 on Aves · Aves 1-hop | ACC: 67.4 (0.3) -> 67.6 (0.3) | No clear effect |
| add hierarchy-adversarial training (ablation) | ResNet-50 on Insecta · Insecta 1-hop | ACC: 82.9 (0.3) -> 82.7 (0.3) | No clear effect |
| add hierarchy-supportive supervision (ablation) | ResNet-50 on Aves · Aves 1-hop | AUROC: 70.1 -> 71.1 | Helped |
| add hierarchy-supportive supervision (ablation) | ResNet-50 on Insecta · Insecta 1-hop | AUROC: 78.0 -> 79.2 | Helped |
| add hierarchy-supportive supervision (ablation) | ResNet-50 on Aves · Aves 1-hop | ACC: 67.4 (0.3) -> 66.6 (0.4) | Hurt |
| add hierarchy-supportive supervision (ablation) | ResNet-50 on Insecta · Insecta 1-hop | ACC: 82.9 (0.3) -> 82.3 (0.1) | Hurt |
| train on finer rather than matching label granularity (ablation) | Aves recognition models · Aves 2-hop to 4-hop open-set | AUROC | Helped |
| use hierarchy-supportive learning (ablation) | OSR models · Coarse-grained OSR | AUROC | Helped |
Surprising findings
- Hierarchy-adversarial learning improves fine-grained OSR while consistently impairing coarser OSR performance.
- Training on overly fine-grained labels yields higher closed-set accuracy at coarser granularities than training at matching granularity, contradicting prior object detection and localization findings.
Limitations
- Relies on taxonomy as proxy for semantic similarity; only tested on birds and insects from iNat2021.
- Hierarchy-adversarial weight requires tuning; too high a value degrades closed-set and open-set performance.
- Method requires hierarchical labels and cannot use attribute labels.
Experimental setup
Datasets: iNat2021
Benchmarks: iNat2021-OSR (Aves, Insecta)
Metrics: AUROC AUPR closed-set accuracy macro accuracy
Baselines: Maximum logit score KL-disagreement Nearest neighbour Cross-entropy Hierarchy-supportive learning
Builds on: Vaze et al. open-set recognition baseline (maximum logit score) gradient reversal layer domain adaptation hierarchical multi-task learning Rodriguez et al. zero-shot taxonomy hops
Closest papers by meaning
- Exploring Diverse Representations for Open Set Recognition (AAAI 2024 · similarity 0.88)
- GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection (NeurIPS 2025 · similarity 0.87)
- Improving Open Set Recognition via Visual Prompts Distilled from Common-Sense Knowledge (AAAI 2024 · similarity 0.87)
- All Beings Are Equal in Open Set Recognition (AAAI 2024 · similarity 0.87)
- Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models (ICLR 2025 · similarity 0.87)
- The Devil is in the Fine-Grained Details: Evaluating Open-Vocabulary Object Detectors for Fine-Grained Understanding (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.