Research map / Label-efficient and robust learning / Domain adaptation and generalization
Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Survey/overview of model reprogramming, which repurposes a frozen pre-trained source model for target-domain tasks by training only input transformation and output mapping layers, with use cases and a theoretical error analysis.
Problem
Deep learning in resource-limited, cross-domain settings faces limited target data, constrained model development cost, and lack of suitable pre-trained models for finetuning.
Approach
Frame model reprogramming as adding a trainable input transformation layer and output mapping layer around a frozen pre-trained source model, enabling cross-domain reuse without finetuning the source model's parameters, and organize existing works and a Wasserstein-based theoretical view of it.
Key results
- Voice2Series outperforms or ties the best baseline on 19 of 30 UCR time-series datasets.
- WARP attains performance comparable to modern language models on GLUE with far fewer trainable parameters (as reported in the cited work).
- Black-box adversarial reprogramming reprograms commercial APIs for bio-medical tasks at roughly 20 US dollars cost (as reported in the cited work).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Voice2Series (V2S) to reprogram a speech model for time-series classification | UCR time-series classification · UCR time-series classification datasets | number of datasets matching or outperforming the best baseline: 19 out of 30 datasets | Mixed |
| Use representation reprogramming via dictionary learning (R2DL) instead of training from scratch | pre-trained English language models (e.g., BERT and LSTM) · reduced-data biochemical sequence classification | performance | Helped |
| Use frequency-based greedy label mapping instead of random label mapping | model reprogramming · classification tasks | performance | Helped |
| Use iterative greedy label mapping | model reprogramming · classification tasks | performance | Helped |
| Assign more non-overlapping source labels to each target label | model reprogramming · classification tasks | performance | Helped |
| Jointly reprogram and fine-tune selected model parameters | pre-trained models · low-resource speech command recognition and music genre classification | performance | Helped |
| Use model reprogramming instead of standard transfer learning | differentially private training · centralized and federated learning | privacy-utility tradeoff | Helped |
Limitations
- Theoretical analysis relies on assumptions such as one-to-one label mapping and reports only numerical evidence from cited work.
- Adversarial reprogramming raises misuse concerns when an attacker repurposes a model without the provider's knowledge.
Experimental setup
Datasets: UCR time series classification datasets CIFAR-10 MNIST GLUE SuperGLUE
Benchmarks: UCR time series classification datasets GLUE
Metrics: accuracy
Baselines: transfer learning training from scratch
Builds on: adversarial reprogramming transfer learning visual prompting word-level adversarial reprogramming
Closest papers by meaning
- DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing (AAAI 2024 · similarity 0.85)
- LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty (CVPR 2025 · similarity 0.85)
- Understanding Model Reprogramming for CLIP via Decoupling Visual Prompts (ICML 2025 · similarity 0.85)
- RESQUE: Quantifying Estimator to Task and Distribution Shift for Sustainable Model Reusability (AAAI 2025 · similarity 0.85)
- Cross-model Control: Improving Multiple Large Language Models in One-time Training (NeurIPS 2024 · similarity 0.84)
- Robustness Reprogramming for Representation Learning (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.