Research map / Architectures and efficiency / Attention and transformers
Meta-Attention for ViT-Backed Continual Learning
Mengqi Xue, Haofei Zhang, Jie Song, Mingli Song
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MEAT instead of the strongest listed continual-learning baseline | DeiT-Ti · CUB, Cars, FGVC-Aircraft, WikiArt, Sketches, and CIFAR-100 | accuracy (%) | Improved |
| Use MEAT instead of the strongest listed continual-learning baseline | DeiT-S · CUB, Cars, FGVC-Aircraft, WikiArt, Sketches, and CIFAR-100 | accuracy (%) | Improved |
| Use MEAT instead of the strongest listed continual-learning baseline | T2T-ViT-12 · CUB, Cars, FGVC-Aircraft, WikiArt, Sketches, and CIFAR-100 | accuracy (%) | Improved |
| Use MEAT instead of Piggyback | DeiT-Ti and DeiT-S · Places365 | accuracy (%) | Improved |
| Add MEAT token masks in MHSA with drop-control loss to the classifier baseline (ablation) | ViT continual learning · Six new-task benchmarks | accuracy (%): +2.67%~7.02% | Improved |
| Add drop-control loss to MHSA token masking (ablation) | ViT continual learning · Six new-task benchmarks | accuracy (%) | Improved |
| Add MEAT masks to FFN neurons (ablation) | ViT continual learning · Six new-task benchmarks | accuracy (%) | Improved |
| Combine MHSA token masks and FFN neuron masks (ablation) | ViT continual learning · Six new-task benchmarks | accuracy (%) | Improved |
| Apply MEAT-style masks to CNN parameters without self-attention masks (ablation) | CNNs · CIFAR-100 | accuracy (%) | No clear effect |
Closest papers by meaning
- Exemplar-Free Continual Learning of Vision Transformers via Gated Class-Attention and Cascaded Feature Drift Compensation (IJCV 2025 · similarity 0.91)
- Continual Learning With Lifelong Vision Transformer (CVPR 2022 · similarity 0.90)
- Task-Free Dynamic Sparse Vision Transformer for Continual Learning (AAAI 2024 · similarity 0.90)
- Visual Prompt Tuning in Null Space for Continual Learning (NeurIPS 2024 · similarity 0.87)
- Revisiting Continuity of Image Tokens for Cross-domain Few-shot Learning (ICML 2025 · similarity 0.86)
- Convolutional Prompting meets Language Models for Continual Learning (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
