Research map / Label-efficient and robust learning / Recognition and long-tail

Enhancing Instance-Level Image Classification with Set-Level Labels

ICLR 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Partial and multi-label learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FACILE, which pretrains instance feature maps on set-level coarse-grained labels (via set-input models) and then fine-tunes fine-grained classifiers on few-shot support sets, with a theoretical fast-rate excess-risk analysis and experiments on CIFAR-100, histopathology (TCGA/GTEx pretraining; LC, PAIP, NCT, PDAC), CUB200 and DINO V2/CLIP backbones.

Problem

Instance-level image classification usually relies on single-instance labels, which are expensive to obtain; the paper asks whether cheaper set-level coarse-grained labels over sets of instances can improve fine-grained few-shot classification.

Approach

Learn an instance feature map by training a set-input model (e.g., Set Transformer) to predict set-level coarse labels from sets of instance embeddings, then freeze the feature map and train a fine-grained classifier on few-shot support sets; the framework is generic and instantiated with FSP and SupCon pretraining, with a theory giving fast O(1/n) excess-risk rates under a relative Lipschitz assu

Key results

• On 5-shot 5-way LC test, FACILE-FSP reaches 90.67 F1 (NC) versus 85.12 for the best self-supervised baseline SimSiam, with roughly 13% gain over the strongest baseline on NCT-pretrained comparisons as stated.
• On CIFAR-100 5-shot 5-way with unique superclass-number pretraining, FACILE-FSP attains 86.25 F1 versus 78.15 for SimSiam.
• Excess risk theorem gives O(1/n) fast rate when m = Omega(n^2) under a 1/sqrt(m) coarse-label rate; experiments show log-linear error trends consistent with this.

Measured effects reported in the paper

Surprising findings

• SimCLR underperforms SimSiam on histopathology, which the authors attribute to the small batch size used for SimCLR.
• DINO V2 ViT-B/14 generalizes poorly to histopathology images, performing below several models fine-tuned from ResNet18 on TCGA-pretrained features in some settings.

Limitations

• Separate use of loosely related coarse and fine labels may be inefficient.
• Pretraining with large coarse-grained datasets and set-input models can be expensive.
• Theoretical guarantees rely on assumptions such as relative Lipschitzness, central condition, and a good embedding existing.

Experimental setup

Datasets: CIFAR-100 TCGA GTEx CUB200 LC25000 PAIP NCT-CRC-HE PDAC (private)

Benchmarks: LC25000 PAIP NCT-CRC-HE CIFAR-100 (few-shot meta-test) CUB200 (anomaly detection finetuning) PDAC

Metrics: F1 accuracy 95% confidence interval

Baselines: SimSiam SimCLR FSP-Patch ImageNet (FSP) pretrained ResNet18 MoCo v3 CLIP (ViT-B/16) DINO V2 (ViT-B/14)

Builds on: SupCon SimSiam SimCLR Latent augmentation

Closest papers by meaning

• Minimizing Labeling Cost for Nuclei Instance Segmentation and Classification with Cross-domain Images and Weak Labels (AAAI 2021 · similarity 0.85)
• Prototype-Based Image Prompting for Weakly Supervised Histopathological Image Segmentation (CVPR 2025 · similarity 0.84)
• Dynamic Policy-Driven Adaptive Multi-Instance Learning for Whole Slide Image Classification (CVPR 2024 · similarity 0.84)
• Fine-grained Classes and How to Find Them (ICML 2024 · similarity 0.83)
• Systematic Comparison of Semi-supervised and Self-supervised Learning for Medical Image Classification (CVPR 2024 · similarity 0.83)
• MAPLE: Multi-scale Attribute-enhanced Prompt Learning for Few-shot Whole Slide Image Classification (NeurIPS 2025 · similarity 0.83)