Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Few-Shot Learning via Repurposing Ensemble of Black-Box Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes black-box model repurposing (MFR) that learns flow-based input augmentations for an ensemble of frozen black-box models, with gradients estimated by finite differences, to solve few-shot target tasks.
Problem
Few-shot learning on a target task when only black-box, non-modifiable pre-trained models of related tasks (accessible via input-output APIs) are available and no source training data or parameters can be accessed.
Approach
Learn invertible flow-based input augmentations for each black-box model, feed augmented inputs to the black boxes, and combine their outputs into a feature ensemble within a VAE-style ELBO that maximizes target-label likelihood; gradients through black boxes are estimated with random directional finite differences.
Key results
- Across 120 3-way tasks per dataset (MNIST, CIFAR-10, Mini-ImageNet), repurposed models outperform the scratch base model in the majority of cases (success cases far outnumber failures), per the bubble plots.
- On CelebA sub-domain repurposing, the proposed method outperforms BE, FBE, BAR and the base model across HAIR, EYE and SKIN categories.
- On Retinopathy Diabetic with ImageNet-pretrained ResNet-18 black boxes, repurposed ensembles significantly outperform scratch models on 20-shot 3/4/5-way tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| repurpose the black-box ensemble with learned input augmentation | Base Model · MNIST, CIFAR-10, Mini-ImageNet 3-way tasks | classification performance | Mixed |
| repurpose black-box models from HAIR, EYE, or SKIN source tasks | Base Model and ensemble baselines · CelebA cross-category tasks | classification performance | Helped |
| repurpose CIFAR-10 black-box models for CelebA target tasks | Base Model and ensemble baselines · CelebA EYE, SKIN, and HAIR tasks | classification performance | Helped |
| repurpose an ensemble of ImageNet-trained 2-way black-box models | scratch-trained model · Retinopathy Diabetic 20-shot 3-way tasks | classification performance | Helped |
| repurpose existing black-box models for few-shot tasks | scratch-trained model · Retinopathy Diabetic 3-way, 4-way, and 5-way tasks | classification performance | Helped |
| use adversarial black-box reprogramming instead of the proposed repurposing method | few-shot repurposing · CelebA sub-domain tasks | classification performance | Hurt |
Surprising findings
- Adversarial reprogramming (BAR) performed worst among all baselines, including the base model, in the CelebA experiments.
Limitations
- Few-shot accuracy is weaker than state-of-the-art meta-learning results, though the setting is harder (no access to source labeled data).
- Some target tasks fail to improve over the base model (red bubbles in Fig. 2).
- BAR baseline was re-implemented because released code was limited to binary tasks with a single source.
Experimental setup
Datasets: MNIST CIFAR-10 Mini-ImageNet CelebA Retinopathy Diabetic
Benchmarks: MNIST 3-way few-shot CIFAR-10 3-way few-shot Mini-ImageNet 3-way few-shot CelebA sub-domain repurposing Retinopathy Diabetic 3/4/5-way 20-shot
Metrics: classification accuracy relative performance gain over base model
Baselines: Base Model (scratch) Black-Box Ensemble (BE) Fine-tuned Black-Box Ensemble (FBE) Black-Box Adversarial Re-programmer (BAR)
Builds on: Tsai et al. 2020 black-box reprogramming (BAR) Hoang et al. 2020 black-box model fusion VAE normalizing flows (planar/radial)
Closest papers by meaning
- Unlocking the Potential of Black-box Pre-trained GNNs for Graph Few-shot Learning (AAAI 2025 · similarity 0.85)
- Black-Box Forgetting (NeurIPS 2024 · similarity 0.85)
- Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning (CVPR 2025 · similarity 0.85)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.84)
- Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages (CVPR 2025 · similarity 0.84)
- H-ensemble: An Information Theoretic Approach to Reliable Few-Shot Multi-Source-Free Transfer (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.