Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Few-Shot Learning from Augmented Label-Uncertain Queries in Bongard-HOI
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes label-uncertain query augmentation (background-blended and rotation) plus a mean-teacher pseudo-labeling scheme with negative-support augmentation for few-shot HOI classification on Bongard-HOI, reaching 68.74% average accuracy.
Problem
Few-shot human-object interaction classification on Bongard-HOI, where positive and negative classes differ only in action and query images can be visually similar to other classes, with only 12 support images per task and no text labels.
Approach
Augment queries without preserving labels (background-blended generation via Unicontrol and 90-degree rotation), treat augmented queries as label-uncertain, and use a mean teacher to produce pseudo-labels for them, with the student seeing rotated negative support images while the teacher sees the original support.
Key results
- Achieves 68.74% average accuracy on Bongard-HOI, versus prior SOTA TPT at 66.59%.
- Reaches 73.27% (73.28% in text) on HICO-FS 5-way 5-shot, versus 71.20% for Meta-baseline.
- Ablation: rotated queries with mean teacher raise average accuracy by 2.06%; adding background-blended queries reaches 68.74%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed method with Our encoder instead of TPT | Bongard-HOI · Bongard-HOI test sets, Avg. | accuracy: increased average accuracy of 2.15% | Helped |
| Add proposed augmentations to Meta-baseline | Meta-baseline on HICO-FS · HICO-FS 5-way 1-shot | accuracy: 58.79 ± 0.22 -> 60.59 ± 0.22 | Helped |
| Add proposed augmentations to Meta-baseline | Meta-baseline on HICO-FS · HICO-FS 5-way 5-shot | accuracy: 71.20 ± 0.21 -> 73.28 ± 0.20 | Helped |
| Use NDA augmentation instead of the proposed query augmentations (ablation) | Bongard-HOI query learning · Bongard-HOI test sets, Avg. | accuracy: 65.97 -> 65.30 | Hurt |
| Add rotated queries with mean-teacher pseudo-label generation (ablation) | Bongard-HOI query learning · Bongard-HOI test sets, Avg. | accuracy: 65.97 -> 68.03; 2.06% | Helped |
| Add background-blended queries to rotated queries (ablation) | Bongard-HOI query learning · Bongard-HOI test sets, Avg. | accuracy: 68.03 -> 68.74 | Helped |
| Use the proposed negative support augmentation design instead of no augmentation (ablation) | Bongard-HOI student model · Bongard-HOI test sets, Avg. | accuracy: 65.97 -> 68.74; 2.77% | Helped |
| Augment all samples with the same HOI label (ablation) | Bongard-HOI student model · Bongard-HOI test sets, Avg. | accuracy: 65.97 -> 66.67 | Helped |
| Augment one randomly selected sample per HOI class (ablation) | Bongard-HOI student model · Bongard-HOI test sets, Avg. | accuracy: 65.97 -> 67.45 | Helped |
| Augment each same-label sample with 0.5 probability (ablation) | Bongard-HOI student model · Bongard-HOI test sets, Avg. | accuracy: 65.97 -> 67.89 | Helped |
Surprising findings
- Naive NDA negative augmentation decreases performance (65.30% vs 65.97% without augmentation).
- About 72.5% of background-blended generated images shifted to a different HOI class, so they were treated as label-uncertain.
Limitations
- Label-uncertainty rates (12.5% for rotation, 72.5% for background blending) are estimated from small manual samples of 200 images.
- Background-blended generation depends on an external generator (Unicontrol) and CLIP matching scores.
Experimental setup
Datasets: Bongard-HOI HICO-DET HICO-FS ImageNet
Benchmarks: Bongard-HOI HICO-FS
Metrics: classification accuracy
Baselines: ANIL Meta-baseline DSN HOITrans TPT Matching Network ProtoNet Relation Network LGM-Net SADG-Net NDA
Builds on: Bongard-HOI DSN Mean Teacher Unicontrol CLIP
Closest papers by meaning
- EZ-HOI: VLM Adaptation via Guided Prompt Learning for Zero-Shot HOI Detection (NeurIPS 2024 · similarity 0.87)
- Exploring Pose-Aware Human-Object Interaction via Hybrid Learning (CVPR 2024 · similarity 0.86)
- Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning (CVPR 2025 · similarity 0.86)
- Multi-Label Few-Shot Image Classification via Pairwise Feature Augmentation and Flexible Prompt Learning (AAAI 2025 · similarity 0.86)
- Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detection (CVPR 2024 · similarity 0.86)
- Visual Diversity and Region-aware Prompt Learning for Zero-shot HOI Detection (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.