Research map / Architectures and efficiency / Pruning and efficient tuning
DELTA: DEEP LEARNING TRANSFER USING FEATURE MAP WITH ATTENTION FOR CONVOLUTIONAL NETWORKS
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use attention-weighted feature-map regularization instead of weight-distance regularization | ResNet-101 transfer learning · Caltech 256-30 | top-1 accuracy: 85.4 ±0.2 -> 86.6±0.1 | Helped |
| use attention-weighted feature-map regularization instead of weight-distance regularization | Inception-V3 transfer learning · Food-101 | top-1 accuracy: 75.9±0.2 -> 80.8±0.2 | Helped |
| add supervised attention to feature-map regularization (ablation) | DELTA with ResNet-101 · CUB-200-2011 | top-1 accuracy: 78.9±0.1 -> 80.5±0.1 | Helped |
| add supervised attention to feature-map regularization (ablation) | DELTA with Inception-V3 · Food-101 | top-1 accuracy: 76.2±0.2 -> 80.8±0.2 | Helped |
| add supervised attention to feature-map regularization (ablation) | DELTA with Inception-V3 · Stanford Dogs 120 | top-1 accuracy: 88.7±0.1 -> 88.7±0.1 | No clear effect |
| apply aspect-ratio-preserving resize and 10-crop testing | ResNet-101 transfer methods · Stanford Dogs 120 | top-1 accuracy: +2.5 for L², +2.5 for L²-SP, +2.5 for DELTA | Helped |
| use DELTA rather than L²-SP | Inception-V3 transfer learning · Stanford Dogs 120 with data augmentation | top-1 accuracy: 91.9±0.1 -> 92.1±0.1 | Helped |
Closest papers by meaning
- Pay Attention to Features, Transfer Learn Faster CNNs (ICLR 2020 · similarity 0.85)
- Co-Tuning for Transfer Learning (NeurIPS 2020 · similarity 0.85)
- Transfer Learning in Infinite Width Feature Learning Networks (ICLR 2026 · similarity 0.85)
- On the Surprising Effectiveness of Attention Transfer for Vision Transformers (NeurIPS 2024 · similarity 0.85)
- Features are fate: a theory of transfer learning in high-dimensional regression (ICML 2025 · similarity 0.84)
- Improving Adversarial Transferability on Vision Transformers via Forward Propagation Refinement (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.