Research map / Detection and segmentation / Open-set and few-shot detection
SimLTD: Simple Supervised and Semi-Supervised Long-Tailed Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SimLTD, a three-step head-to-tail transfer pipeline (pre-train on head classes, transfer to tail classes, fine-tune on sampled head+tail shots) with optional unlabeled images and pseudo-labeling, achieving new LVIS v1 results without extra image labels.
Problem
Long-tailed object detection on LVIS v1, where rare classes have very few training exemplars; prior methods depend on large external labeled databases (ImageNet) or complex multi-stage distillation/meta-learning.
Approach
A simple three-stage head-to-tail transfer: pre-train detector on head classes (optionally semi-supervised with unlabeled images), copy head representations and train only the classifier/regressor on tail classes with copy-pasted rare instances in unlabeled images, then fuse head and tail heads and fine-tune on a 30-shot-per-class sample, without distillation or meta-learning.
Key results
- SimLTD MixPL with DINO Swin-B on COCO-unlabeled reaches 49.0 mAPbox and 43.4 APr on LVIS v1 val, versus RichSem 48.2 mAPbox and 46.5 APr with CLIP and ImageNet-21K labels.
- SimLTD supervised Faster R-CNN R50-FPN reaches 29.0 mAPbox and 20.9 APr, outperforming prior related detectors without external data.
- Transfer learning on tail classes adds up to +4.7 APr and cuts fine-tuning iterations by about two-thirds.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SimLTD's supervised multi-stage training instead of Dong et al.'s multi-stage method | Deformable DETR, R50 · LVIS v1 validation | mAPbox: 28.7 -> 35.0 | Helped |
| Use SimLTD's supervised multi-stage training instead of Dong et al.'s multi-stage method | Deformable DETR, R50 · LVIS v1 validation | APr: 21.8 -> 32.0 | Helped |
| Use SimLTD supervised training instead of RichSem without external data | Swin backbones · LVIS v1 validation | APr: +6.0 APr | Helped |
| Use SimLTD MixTeacher instead of CascadeMatch | LVIS v1 validation, Faster R-CNN R101-FPN · LVIS v1 validation | APfixed box: 32.9 -> 34.4 | Helped |
| Use DeformDETR instead of Faster R-CNN | SimLTD on Objects365 v2 · Objects365 v2 validation | mAPbox: 17.8 -> 27.3 | Helped |
| Use multi-stage training rather than single-stage training on the full long-tailed dataset (ablation) | SimLTD supervised baseline · LVIS v1 validation | APr: 16.8 -> 20.9 | Helped |
| Add photometric jittering (ablation) | Multi-stage SimLTD supervised baseline · LVIS v1 validation | APr: 17.9 -> 18.2 | Helped |
| Add Repeat Factor Sampling (ablation) | Multi-stage SimLTD supervised baseline · LVIS v1 validation | APr: 18.2 -> 19.3 | Helped |
| Add Simple Copy-Paste (ablation) | Multi-stage SimLTD supervised baseline · LVIS v1 validation | mAPbox: 27.1 -> 29.0 | Helped |
| Add Simple Copy-Paste (ablation) | Multi-stage SimLTD supervised baseline · LVIS v1 validation | APr: 19.3 -> 20.9 | Helped |
| Transfer head representations to tail classes instead of random initialization (ablation) | SimLTD tail model · LVIS v1 validation | APr: +4.7 APr | Helped |
| Transfer head representations to tail classes instead of random initialization (ablation) | SimLTD tail model · LVIS v1 validation | fine-tuning iterations: shortening the fine-tuning time by 2/3 | Helped |
Surprising findings
- Training from scratch on tail classes yields roughly half the accuracy of transferred COCO representations, even though COCO differs from LVIS in scope and size.
- Naively applying unlabeled data can worsen pseudo-label class imbalance because unlabeled scenes follow a similar long-tailed distribution.
Limitations
- Relies on the COCO-tuned hyperparameters without re-tuning for LVIS semi-supervised setting
- Fine-tuning shot count (30) is dataset-dependent and chosen by a knee-in-curve analysis
- Gains on APr with Swin backbones are slightly below RichSem
Experimental setup
Datasets: LVIS v1 COCO-unlabeled2017 Objects365
Benchmarks: LVIS v1 Objects365 v2
Metrics: mAPbox APr APc APf APfixed
Baselines: Seesaw Loss NorCal AHRL EFL LogN Dong et al. Detic RichSem DiffusionDet MosaicOS CascadeMatch FASA
Builds on: Soft Teacher MixTeacher MixPL Simple Copy-Paste Repeat Factor Sampling Deformable DETR DINO
Closest papers by meaning
- Adaptive Hierarchical Representation Learning for Long-Tailed Object Detection (CVPR 2022 · similarity 0.91)
- Learning from Rich Semantics and Coarse Locations for Long-tailed Object Detection (NeurIPS 2023 · similarity 0.91)
- Equalization Loss for Long-Tailed Object Recognition (CVPR 2020 · similarity 0.89)
- Long-tailed Object Detection Pretraining: Dynamic Rebalancing Contrastive Learning with Dual Reconstruction (NeurIPS 2024 · similarity 0.88)
- OFBD: Object-Focused Background Debiasing for Long-Tailed Learning (NeurIPS 2026 · similarity 0.87)
- Search and Detect: Training-Free Long Tail Object Detection via Web-Image Retrieval (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.