Research map / Detection and segmentation / Open-set and few-shot detection
HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debugging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HiBug2, a closed-loop framework that generates task-specific visual attributes with GPT-4V, enumerates error slices efficiently via tree-structured pruning and intersection, and predicts unseen error slices for model repair.
Problem
Discovering coherent, interpretable error slices in vision models is hard because attribute annotations are missing, attribute combinations explode combinatorially, and validation sets miss high-risk slices.
Approach
Structured task-specific attribute generation (main object, background, global) with comparative image-pair prompting and multi-stage tag refinement, combined with a breadth-first tree enumeration that prunes by data count and intersects matched parent slices, plus tag substitution and GPT-based instruction methods to predict unseen slices.
Key results
- Slice enumeration achieves about 115x and 510x speedups over naive enumeration for 3- and 4-attribute slices, and 12x and 7x over the tree-structured baseline.
- Model repair on classification reaches 0.839 accuracy (+7.6%) versus 0.832 for HiBug and 0.817 for random selection, from an original 0.780.
- Predicted unseen slices cause performance drops of up to 64.6% (tag substitution, object detection) and 34.5% (pose estimation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select repair data using HiBug2-discovered error slices instead of HiBug | Model repair · Image classification | accuracy: 0.832 (+6.3%) -> 0.839 (+7.6%) | Helped |
| Select repair data using HiBug2-discovered error slices instead of random selection | Model repair · Pose estimation | keypoint AP: 0.829 (+0.9%) -> 0.839 (+2.1%) | Helped |
| Select repair data using HiBug2-discovered error slices instead of HiBug | Model repair · Object detection | mAP: 0.673 (+3.4%) -> 0.683 (+4.9%) | Helped |
| Use Tag Substitution to predict error slices beyond the validation set | HiBug2 · Image classification predicted slices | performance degradation: -7.6% | Helped |
| Use Tag Substitution to predict error slices beyond the validation set | HiBug2 · Pose estimation predicted slices | performance degradation: -34.5% | Helped |
| Use Tag Substitution to predict error slices beyond the validation set | HiBug2 · Object detection predicted slices | performance degradation: -64.6% | Helped |
| Use instruction-based prediction to find error slices beyond the validation set | HiBug2 · Image classification predicted slices | performance degradation: -18.4% | Helped |
| Use instruction-based prediction to find error slices beyond the validation set | HiBug2 · Pose estimation predicted slices | performance degradation: -27.2% | Helped |
| Use instruction-based prediction to find error slices beyond the validation set | HiBug2 · Object detection predicted slices | performance degradation: -18.0% | Helped |
| Add pruning and matched-pair intersection to the tree-structured slice enumeration algorithm (ablation) | Tree-structured slice enumeration · Slice enumeration with 3 attributes | speedup: 12x | Helped |
| Add pruning and matched-pair intersection to the tree-structured slice enumeration algorithm (ablation) | Tree-structured slice enumeration · Slice enumeration with 4 attributes | speedup: 7x | Helped |
| Fine-tune on data matching one identified error slice | ResNet · Image classification, matching error slice | accuracy: 0.500 -> 0.875 | Helped |
Surprising findings
- Top 10% error slices overlap 86% across object detection models, versus only 31% across classification models, suggesting classification failures are driven more by data distribution issues.
Limitations
- Relies on GPT-4V/GPT-4 for attribute and tag assignment, which can produce tagging errors.
- Error slices lack ground truth, making evaluation difficult and comparisons across methods hard to standardize.
- User study involves only 4 participants and classification only.
Experimental setup
Datasets: ImageNet COCO KITTI private industrial pose dataset
Benchmarks: ImageNet bear species private pose dataset KITTI
Metrics: accuracy OKS-based keypoint AP mAP IoU slice coherence (user study) overlap of top 10% slices performance degradation on predicted slices
Baselines: HiBug Domino AdaVision Naive slice enumeration Tree-structured enumeration baseline Random selection
Builds on: HiBug AdaVision Domino CLIP
Closest papers by meaning
- HiBug: On Human-Interpretable Model Debug (NeurIPS 2023 · similarity 0.91)
- DD-CAM: Minimal Sufficient Explanations for Vision Models Using Delta Debugging (NeurIPS 2026 · similarity 0.85)
- Slice3D: Multi-Slice Occlusion-Revealing Single View 3D Reconstruction (CVPR 2024 · similarity 0.82)
- SPADE: Sparsity-Guided Debugging for Deep Neural Networks (ICML 2024 · similarity 0.82)
- Beyond Accuracy: Ensuring Correct Predictions With Correct Rationales (NeurIPS 2024 · similarity 0.82)
- Evaluating Model Bias Requires Characterizing its Mistakes (ICML 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.