Research map / Detection and segmentation / Weakly supervised detection and segmentation
As Pseudo-Label Free as Possible: Leveraging Adaptive Feature Generation for Sparsely Annotated Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdaptFG, a pseudo-label-free sparsely annotated object detection method that generates RoI features from class names using a CLIP-infused VAE plus an adaptor, and adds prototypical metric learning and class-relation reasoning on top of SparseDet.
Problem
Sparsely annotated object detection suffers performance decline from insufficient feature diversity, and pseudo-labeling methods introduce noise under extreme sparsity.
Approach
Use a CLIP-infused VAE conditioned on class-name embeddings, with a learned MLP adaptor mapping generated features into the detector's RoI feature space, to synthesize class-balanced features without pseudo-labels; add prototypical metric learning and self-attention-based inter-class relation reasoning to the classifier.
Key results
- On COCO Split-1 at 70% sparsity, AdaptFG reaches 38.2 AP? vs SparseDet 36.4 (Table 1 reports 36.4 for AdaptFG at 70% Split-1 column ordering; text states +3.9% over SparseDet at 70% Split-1).
- On PASCAL VOC Split-4 (Extreme), AdaptFG achieves 78.1 AP50 vs SparseDet 75.0 (+3.1).
- On PASCAL VOC Split-4 Easy/Hard, AdaptFG reaches 82.9/82.5 AP50 vs SparseDet 81.6/80.9.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SparseDet with AdaptFG | Faster R-CNN ResNet-101 · COCO Splits 1–3, 30%–70% annotation removal | AP: AdaptFG outperforms SparseDet across all metrics | Helped |
| replace SparseDet with AdaptFG | Faster R-CNN ResNet-101 · PASCAL VOC Splits 4–5 | AP50: 3.1% (Extreme); 2.8% (50%) | Helped |
| add prototypical metric learning (ablation) | SparseDet on PASCAL VOC Split-4 Extreme · PASCAL VOC Split-4 Extreme | AP50: 75.0 -> 75.7 | Helped |
| add synthetic feature integration after PL (ablation) | SparseDet on PASCAL VOC Split-4 Extreme · PASCAL VOC Split-4 Extreme | AP50: 75.7 -> 76.3 | Helped |
| add class-relation reasoning to PL and FI (ablation) | SparseDet on PASCAL VOC Split-4 Extreme · PASCAL VOC Split-4 Extreme | AP50: 76.3 -> 77.2 | Helped |
| add the Adaptor to PL and FI (ablation) | SparseDet on PASCAL VOC Split-4 Extreme · PASCAL VOC Split-4 Extreme | AP50: 76.3 -> 76.8 | Helped |
| use all components (PL, FI, CR, and Adaptor) (ablation) | SparseDet on PASCAL VOC Split-4 Extreme · PASCAL VOC Split-4 Extreme | AP50: 75.0 -> 78.1 | Helped |
| increase synthetic feature samples from 20 to 100 (ablation) | AdaptFG on PASCAL VOC Split-4 · PASCAL VOC Split-4 Easy | AP50: 81.9 -> 82.9 | Helped |
| increase synthetic feature samples from 100 to 200 (ablation) | AdaptFG on PASCAL VOC Split-4 · PASCAL VOC Split-4 Easy | AP50: 82.9 -> 82.6 | Hurt |
Limitations
- Gains are smaller at low sparsity (e.g., 30%) where performance is already close to the oracle.
- Built on top of SparseDet, so it depends on a pseudo-label-free baseline's unlabeled-region discovery component.
Experimental setup
Datasets: MS COCO PASCAL VOC 2007+2012
Benchmarks: MS COCO PASCAL VOC
Metrics: AP AP50
Baselines: Pseudo Label BRL Unbiased Teacher Co-mining SparseDet Efficient-FSOD
Builds on: SparseDet Norm-VAE CLIP Faster R-CNN CoOp-style learnable prompts
Closest papers by meaning
- Sparse Semi-DETR: Sparse Learnable Queries for Semi-Supervised Object Detection (CVPR 2024 · similarity 0.86)
- Taming Self-Training for Open-Vocabulary Object Detection (CVPR 2024 · similarity 0.86)
- Generative Region-Language Pretraining for Open-Ended Object Detection (CVPR 2024 · similarity 0.86)
- HINTED: Hard Instance Enhanced Detector with Mixed-Density Feature Fusion for Sparsely-Supervised 3D Object Detection (CVPR 2024 · similarity 0.85)
- Object Detection Benchmarks are Incomplete: The Role of Label Errors and Annotation Uncertainty (NeurIPS 2026 · similarity 0.85)
- Test-Time Adaptive Object Detection with Foundation Model (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.