Research map / Detection and segmentation / CNN features
Bounding Box Stability against Feature Dropout Reflects Detector Generalization across Environments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the box stability score (BoS), the IoU between bipartite-matched boxes with and without MC feature-map dropout, and shows it linearly predicts detector mAP on unlabeled test sets for vehicle and pedestrian detection.
Problem
Estimating object detector accuracy (mAP) and generalization on unlabeled real-world test sets, where ground-truth boxes are unavailable; existing AutoEval methods are designed for classification and do not account for box regression.
Approach
Under MC feature-map dropout, accurate detectors keep their bounding boxes stable. The method matches boxes between the original and dropout-perturbed outputs with the Hungarian algorithm and averages GIoU-based stability over a test set (BoS score). A linear regressor maps BoS to mAP, trained on a meta-set of transformed labeled sample sets.
Key results
- On vehicle detection with leave-one-out over 10 datasets, BoS achieves average RMSE 2.25% versus 4.46% for the second-best method ATC.
- BoS-mAP correlation on synthesized sample sets reaches R^2 > 0.94 and Spearman rho > 0.93 for RetinaNet+R50 trained on COCO.
- On pedestrian detection, BoS achieves average RMSE 3.29%, the best among compared methods (FD 3.76%, ES 3.42%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BoS instead of the strongest listed baseline, ATC | label-free vehicle mAP estimation with RetinaNet + ResNet-50 · Vehicle detection, 10 unseen datasets | RMSE (%): 4.46 -> 2.25 ±0.13 | Helped |
| use BoS instead of the strongest listed baseline, ES | label-free pedestrian mAP estimation · Pedestrian detection, 9 unseen datasets | RMSE (%): 3.42 -> 3.29 ±0.07 | Helped |
| use BoS instead of the strongest listed baseline, PS | label-free multi-class mAP estimation · Multi-class detection, 5 datasets | RMSE (%): 4.69 -> 4.29 ± 0.70 | Helped |
| add prediction score to BoS (ablation) | label-free vehicle mAP estimation · Vehicle detection, 10 unseen datasets | RMSE (%): 2.25 ±0.13 -> 2.24 ±0.12 | No clear effect |
| add prediction score to BoS (ablation) | label-free pedestrian mAP estimation · Pedestrian detection, 9 unseen datasets | RMSE (%): 3.29 ±0.07 -> 3.48 ±0.07 | Hurt |
| add prediction score to BoS (ablation) | label-free multi-class mAP estimation · Multi-class detection, 5 datasets | RMSE (%): 4.29 ± 0.70 -> 4.90 ± 0.66 | Hurt |
| measure confidence stability rather than bounding-box position stability (ablation) | label-free detector evaluation · Vehicle detection sample sets | correlation with mAP | Hurt |
| use feature dropout with different backbone positions and dropout rates, selecting by training R² (ablation) | BoS · Vehicle detection | R² between mAP and BoS | Mixed |
| increase the number of unlabeled test images (ablation) | BoS-based mAP estimator · Vehicle detection | RMSE (%): ∼2.54% | Helped |
| increase meta-set size or sample-set size (ablation) | BoS-based mAP estimator · Vehicle detection | RMSE (%) | Helped |
Surprising findings
- Confidence stability (softmax change after dropout) correlates poorly with accuracy, while box-position stability correlates strongly.
- Combining BoS with existing confidence-based statistics (PS, ES, ATC, FD) gives no noticeable improvement and sometimes degrades performance.
Limitations
- Correlation weakens when mAP is very low (below about 15%), since too many missed detections make box matching unreliable.
- Correlation is weak for RetinaNet+Swin and Faster R-CNN+R50 and not strong for the Swin backbone, where BoS performs less well.
- BoS is a dataset-level measure and correlates weakly with accuracy on individual images; it is also non-differentiable and cannot serve as a training loss.
Experimental setup
Datasets: COCO BDD Cityscapes DETRAC Exdark Kitti Self-driving Roboflow Udacity Traffic Caltech CrowdHuman
Benchmarks: COCO BDD Cityscapes DETRAC Exdark Kitti Self-driving Roboflow Udacity Traffic Caltech CrowdHuman
Metrics: RMSE R^2 Spearman rank correlation mAP mAP50 mAP75
Baselines: Prediction score (PS) Entropy score (ES) Average confidence (AC) ATC Frechet distance (FD)
Builds on: MC dropout GIoU loss DETR-style Hungarian matching ATC Frechet distance
Closest papers by meaning
- Towards Reliable Detection of Empty Space: Conditional Marked Point Processes for Object Detection (ICLR 2026 · similarity 0.88)
- Bounding Box Regression With Uncertainty for Accurate Object Detection (CVPR 2019 · similarity 0.88)
- Theoretically Achieving Continuous Representation of Oriented Bounding Boxes (CVPR 2024 · similarity 0.86)
- Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection (NeurIPS 2020 · similarity 0.86)
- Improving Single Domain-Generalized Object Detection: A Focus on Diversification and Alignment (CVPR 2024 · similarity 0.84)
- Delving into Cascaded Instability: A Lipschitz Continuity View on Image Restoration and Object Detection Synergy (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.