Research map / Architectures and efficiency / RNNs and normalization
Rapid Adaptation with Conditionally Shifted Neurons
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ResNet rather than CNN as the base architecture with conditional shifts | adaCNN/adaResNet · Mini-ImageNet 5-way 1-shot | accuracy: 48.34 ± 0.68 -> 56.88 ± 0.62 | Helped |
| Use ResNet rather than CNN as the base architecture with conditional shifts | adaCNN/adaResNet · Mini-ImageNet 5-way 5-shot | accuracy: 62.00 ± 0.55 -> 71.94 ± 0.57 | Helped |
| Replace gradient conditioning with direct feedback conditioning | adaResNet · Mini-ImageNet 5-way 1-shot | accuracy: 56.62 ± 0.69 -> 56.88 ± 0.62 | No clear effect |
| Replace gradient conditioning with direct feedback conditioning | adaResNet · Mini-ImageNet 5-way 5-shot | accuracy: 71.69 ± 0.67 -> 71.94 ± 0.57 | Hurt |
| Adapt the recurrent LSTM with task-specific shifts rather than adapting only an output feed-forward network | PTB few-shot models · Penn Treebank 5-way few-shot language modeling | accuracy: 2-layer LSTM + adaFFN (∇): 32.55/33.2, 44.15/46.0, 50.4/51.7; 2-layer adaLSTM (∇): 43.1/43.0, 52.05/54.2, 57.35/58.4 | Helped |
| Increase the adaLSTM depth from one layer to two | adaLSTM · Penn Treebank few-shot language modeling | accuracy: 1-layer adaLSTM (∇): 36.55/37.7, 43.25/44.6, 50.7/52.1; 2-layer adaLSTM (∇): 43.1/43.0, 52.05/54.2, 57.35/58.4 | Helped |
| Add more labeled support examples for the target word | 2-layer adaLSTM · Penn Treebank few-shot language modeling | accuracy: 43.1/43.0 -> 52.05/54.2 -> 57.35/58.4 | Helped |
| Use direct feedback rather than gradient conditioning | 2-layer adaLSTM · Penn Treebank few-shot language modeling | runtime: 200 seconds vs. 160 seconds | Helped |
| Remove the nonlinearity applied to the conditional shift (ablation) | adaCNN and adaLSTM · Mini-ImageNet one-shot and PTB one-shot | accuracy: around 7% | Hurt |
| Add the conditional shift to pre-activations inside the nonlinearity (ablation) | adaCNN and adaLSTM · Mini-ImageNet one-shot and PTB one-shot | accuracy: 47.8%; 48.09%; 30.7/31.8%; 33.25/34.05% | Mixed |
| Replace the learned MLP and gradient preprocessing with a learned scalar multiplying the raw gradient (ablation) | gradient-conditioned adaCNN and adaLSTM · Mini-ImageNet one-shot and PTB one-shot | accuracy: about 3 percentage points; 20% | Hurt |
| Replace the deep mapping function with a one-layer perceptron for direct-feedback information (ablation) | DF-conditioned adaCNN and adaLSTM · Mini-ImageNet and PTB few-shot tasks | accuracy: over 10% | Hurt |
Closest papers by meaning
- Task-Conditioned Adaptation of Visual Features in Multi-Task Policy Learning (CVPR 2024 · similarity 0.85)
- On the Duality between Gradient Transformations and Adapters (ICML 2025 · similarity 0.83)
- PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.83)
- Memory-Integrated Reconfigurable Adapters: A Unified Framework for Settings with Multiple Tasks (NeurIPS 2025 · similarity 0.83)
- Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks (NeurIPS 2025 · similarity 0.83)
- Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.