Research map / Architectures and efficiency / Attention and transformers
Multi-scale Activation, Refinement, and Aggregation: Exploring Diverse Cues for Fine-Grained Bird Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MDCM, an Activation-Selection-Aggregation framework on a multi-scale ViT (MS-ViT) that activates scale-specific cues, selects discriminative tokens per stage, and gate-aggregates per-stage predictions for fine-grained bird recognition.
Problem
Fine-grained bird recognition suffers from subtle inter-species differences, complex backgrounds, and large scale variation; plain ViTs have limited receptive fields and are vulnerable to scale variance, and earlier-stage features are lost in multi-scale ViTs.
Approach
Combine a parameter-free cls-attention-based activation mask that makes cues at different stages mutually distinct (MSCA), a per-stage top-k token selection guided by deeper-stage cls information (MSTS), and a learned sigmoid gating that adaptively fuses the per-stage classifier outputs (MSDA) on top of MS-ViT.
Key results
- Achieves 92.5% top-1 accuracy on CUB-200-2011, above M2Former (92.4%) and TransFG (91.7%).
- Achieves 92.0% on NABirds and 79.8% on iNat2017, versus M2Former's 91.1% and 77.8%.
- Ablation on CUB: MS-ViT baseline 91.6%, adding MSTS 91.9%, adding MSDA 92.3%, adding MSCA 92.5%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MDCM instead of M2Former | MS-ViT-Base · CUB-200-2011 | accuracy: 92.4 -> 92.5 | Helped |
| use MDCM instead of ACC-ViT | bird recognition · NABirds | accuracy: 91.4 -> 92.0 | Helped |
| use MDCM instead of M2Former | MS-ViT-Base · iNat2017 | accuracy: 77.8 -> 79.8 | Helped |
| add MSTS to the MS-ViT baseline (ablation) | MDCM · CUB-200-2011 | accuracy: 91.6 -> 91.9 | Helped |
| add MSDA to MSTS (ablation) | MDCM · CUB-200-2011 | accuracy: 91.9 -> 92.3 | Helped |
| add MSCA to MSTS and MSDA (ablation) | MDCM · CUB-200-2011 | accuracy: 92.3 -> 92.5 | Helped |
| replace addition aggregation with gating (ablation) | MSDA · MSDA aggregation comparison | Score: 113 -> 173 | Helped |
| add Stage 4 cues (ablation) | multi-stage cue recognition · CUB-200-2011 Total | accuracy: 91.6 -> 91.7 | Helped |
| add Stage 3 cues to Stage 4 cues (ablation) | multi-stage cue recognition · CUB-200-2011 Total | accuracy: 91.7 -> 92.0 | Helped |
| add Stage 2 cues (ablation) | multi-stage cue recognition · CUB-200-2011 Total | accuracy: 92.0 -> 92.4 | Helped |
| add Stage 1 cues (ablation) | multi-stage cue recognition · CUB-200-2011 Total | accuracy: 92.4 -> 92.5 | Helped |
Surprising findings
- Simple summation of per-stage predictions is less accurate than gated aggregation (ablation: 113 vs 173 score), though both yield positive correction scores.
Limitations
- Gains over the strongest prior ViT baselines on CUB are small (0.1% over M2Former).
- Ablation on the MSCA module shows only a 0.2% improvement on CUB.
Experimental setup
Datasets: CUB-200-2011 NABirds iNat2017 ImageNet21K
Benchmarks: CUB-200-2011 NABirds iNat2017
Metrics: top-1 accuracy
Baselines: NTS-Net Cross-X DCL PMG MCEN GaRD CMN MA-ASN GDSMP-Net TA-CFN ViT RAMS-Trans
Builds on: MS-ViT M2Former TransFG Swin Transformer label smoothing
Closest papers by meaning
- Prompt-CAM: Making Vision Transformers Interpretable for Fine-Grained Analysis (CVPR 2025 · similarity 0.88)
- RealBirdID: Benchmarking Bird Species Identification in the Era of MLLMs (CVPR 2026 · similarity 0.86)
- BirdCollect: A Comprehensive Benchmark for Analyzing Dense Bird Flock Attributes (AAAI 2024 · similarity 0.85)
- CleverBirds: A Multiple-Choice Benchmark for Fine-grained Human Knowledge Tracing (NeurIPS 2025 · similarity 0.84)
- NetTrack: Tracking Highly Dynamic Objects with a Net (CVPR 2024 · similarity 0.84)
- Democratizing Fine-grained Visual Recognition with Large Language Models (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.