Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning

MetaCoCo: A New Few-Shot Classification Benchmark with Spurious Correlation

ICLR 2024 · Label-efficient and robust learning › Zero-/few-shot and meta-learning · approach: Alignment-based few-shot metric learning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MetaCoCo, a few-shot classification benchmark of 100 concepts and 155 contexts with spurious-correlation shifts, and evaluates many FSC, cross-domain, and self-supervised methods on it, finding substantial performance drops.

Problem

Few-shot classification models degrade under spurious-correlation shifts, where non-causal contexts correlated with labels in base classes do not hold for novel classes, but no FSC benchmark existed to evaluate this.

Approach

Builds a real-world FSC benchmark where each concept is paired with varied contexts (e.g., "dog on grass", "dog in snow"), so base and novel splits can differ in context, and proposes a CLIP-based concept-context similarity metric to quantify spurious correlation strength.

Key results

• Across 17 FSC methods and six self-supervised methods, accuracy on MetaCoCo drops compared with existing FSC benchmarks; e.g., ProtoNet 1-shot 43.14% falls to 40.65% with rotation SSFSL.
• Cross-domain methods (ATA, FT) lose their advantage on MetaCoCo, sometimes performing worse than non-cross-domain FSC methods.
• CLIP-based similarity shows MetaCoCo has lower concept-image similarity (0.142) than existing benchmarks (e.g., 0.211 for miniImageNet).

Measured effects reported in the paper

Surprising findings

• Self-supervised rotation auxiliary losses sometimes hurt few-shot accuracy on MetaCoCo.
• Stronger backbones do not always improve performance under spurious-correlation shifts.

Limitations

• Context labels and filtering rely on manual annotation (~2,500 hours) and crawled image search results, which limits scale.
• Benchmark uses a fixed 64/16/20 concept split and ordering by sample count, which may not capture all spurious-correlation settings.

Experimental setup

Datasets: MetaCoCo mini-ImageNet CUB-200-2011 Meta-dataset BSCD-FSL DomainNet

Benchmarks: MetaCoCo

Metrics: top-1 mean accuracy 5-way 1-shot accuracy 5-way 5-shot accuracy

Baselines: Baseline++ RFS-simple Neg-Cosine ProtoNet MAML MoCo SimCLR IFSL ATA FT

Builds on: Meta-dataset BSCD-FSL DomainNet CLIP MAML

Closest papers by meaning

• Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.89)
• Making Large Vision Language Models to Be Good Few-Shot Learners (AAAI 2025 · similarity 0.88)
• A Unified Few-Shot Classification Benchmark to Compare Transfer and Meta Learning Approaches (NeurIPS 2021 · similarity 0.88)
• Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-Domain Few-Shot Learning (CVPR 2024 · similarity 0.86)
• Scaling Few-Shot Learning for the Open World (AAAI 2024 · similarity 0.86)
• Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning (CVPR 2025 · similarity 0.86)