Research map / Architectures and efficiency / Attention and transformers
START: A Generalized State Space Model with Saliency-Driven Token-Aware Transformation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes START, a saliency-driven token-aware style perturbation for Mamba/VMamba-based state space models that suppresses domain-specific features in input-dependent SSM matrices, improving domain generalization with linear complexity.
Problem
Domain generalization with state space models: input-dependent matrices in Mamba can accumulate and amplify domain-specific features, hurting generalization to unseen domains.
Approach
Identify salient tokens via the SSM's input-dependent matrices (START-M) or input activations (START-X) and apply MixStyle-like mean/variance mixing only to those tokens during training, with no extra parameters and disabled at inference.
Key results
- START-M reaches 91.77% average on PACS vs 89.94% for the VMamba strong baseline and 91.20% for DGMamba.
- START-X reaches 58.27% on TerraIncognita vs 56.16% baseline and 52.40% for the best CNN method (AGFA), per the paper's claim.
- START-M reaches 77.09% on OfficeHome and 81.32% on VLCS, outperforming DGMamba's 80.80%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add START-M to VMamba-T | VMamba-T · PACS | average accuracy: 89.94% vs. 91.77% | Helped |
| Add START-M to VMamba-T | VMamba-T · OfficeHome | average accuracy: 76.43% vs. 77.09% | Helped |
| Add START-M to VMamba-T | VMamba-T · VLCS | average accuracy: 79.42% vs. 81.32% | Helped |
| Add START-X to VMamba-T | VMamba-T · TerraIncognita | average accuracy: 56.16% vs. 58.27% | Helped |
| Add START-X to VMamba-T | VMamba-T · DomainNet | average accuracy: 51.45% vs 52.81% | Helped |
| Add saliency-guided token selection instead of random selection (ablation) | START on VMamba-T · PACS | average accuracy: 90.81{ $\pm$0.24} -> 91.77{ $\pm$0.40} | Helped |
| Select salient tokens instead of perturbing the entire input sequence (ablation) | START on VMamba-T · PACS | average accuracy: 90.94{ $\pm$0.18} -> 91.77{ $\pm$0.40} | Helped |
| Add saliency-guided token selection instead of random selection (ablation) | START-X on VMamba-T · TerraIncognita | average accuracy: 57.30% vs. 58.27% | Helped |
| Select salient tokens instead of perturbing the entire input sequence (ablation) | START-X on VMamba-T · TerraIncognita | average accuracy: 57.44% vs. 58.27% | Helped |
| Add START-X to ViM-T | ViM-T · PACS | average accuracy: 85.08% vs. 86.74% | Helped |
| Add START-X to ViM-S | ViM-S · PACS | average accuracy: 87.38% vs. 89.41% | Helped |
| Add START-M to DeiT-Small | DeiT-Small · PACS | average accuracy: 85.85% vs. 87.05% | Helped |
Surprising findings
- Perturbing all tokens (w/o token selection) and random token selection both still improve over the baseline, but salient-token selection performs best, with START-M degenerating to MixStyle when the token ratio is 1.
Limitations
- Theoretical analysis relies on simplifying assumptions (e.g., approximating S_C ratio to 1, large dimension approximation).
- Perturbation ratio P_token was tuned (0.75 chosen) and START-X is less effective at lower P_token values.
Experimental setup
Datasets: PACS OfficeHome VLCS TerraIncognita DomainNet ImageNet
Benchmarks: PACS OfficeHome VLCS TerraIncognita DomainNet
Metrics: accuracy (average over leave-one-domain-out splits)
Baselines: DeepAll PCL EoA EQRM SAGM iDAG DomainDrop CCFP MADG PGrad AGFA GMDG
Builds on: Mamba VMamba MixStyle DSU ALOFT
Closest papers by meaning
- DGFamba: Learning Flow Factorized State Space for Visual Domain Generalization (AAAI 2025 · similarity 0.90)
- MaskViM: Domain Generalized Semantic Segmentation with State Space Models (AAAI 2025 · similarity 0.88)
- Multimodal Unsupervised Domain Generalization by Retrieving Across the Modality Gap (ICLR 2025 · similarity 0.88)
- PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization (CVPR 2024 · similarity 0.88)
- Domain Generalization in CLIP via Learning with Diverse Text Prompts (CVPR 2025 · similarity 0.87)
- CLIPCEIL: Domain Generalization through CLIP via Channel rEfinement and Image-text aLignment (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.