Research map / Architectures and efficiency / CNN design
Enhancing Feature Diversity Boosts Channel-Adaptive Vision Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiChaViT, a channel-adaptive ViT for multi-channel imaging that adds diverse channel sampling, channel and token diversification losses, and orthogonal initialization to reduce feature redundancy across channels.
Problem
Channel-adaptive vision transformers for multi-channel imaging learn redundant features across channels when trained with random channel masking, hurting robustness and channel-specific feature learning under varying channel configurations at test time.
Approach
Replace uniform random channel sampling with diversity-aware sampling that picks channels dissimilar to an anchor channel, and add a proxy-NCA-style channel diversification loss on channel tokens plus an orthogonality-based token diversification loss on patch tokens, with orthogonal initialization.
Key results
- DiChaViT reaches 69.19% full-channel accuracy on JUMP-CP vs 67.51% for ChannelViT, and 63.36% on So2Sat vs 61.03%.
- On CHAMMI average score, DiChaViT achieves 69.66 vs 64.90 for ChannelViT.
- Removing DCS drops JUMP-CP partial-channel accuracy from 57.98 to 42.37 and So2Sat partial from 47.76 to 17.88.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CDL (ablation) | DiChaViT · CHAMMI | Avg score: 68.07 -> 69.66 | Helped |
| add TDL (ablation) | DiChaViT · CHAMMI | Avg score: 67.61 -> 69.66 | Helped |
| add DCS (ablation) | DiChaViT · So2Sat Partial | accuracy: 17.88 -> 47.76 | Helped |
| add DCS (ablation) | DiChaViT · JUMP-CP Partial | accuracy: 42.37 -> 57.98 | Helped |
| use both same-channel and different-channel terms in TDL rather than only the same-channel term (ablation) | DiChaViT · CHAMMI | Avg score: 65.47 -> 69.66 | Helped |
| use both same-channel and different-channel terms in TDL rather than only the different-channel term (ablation) | DiChaViT · So2Sat | accuracy: 62.50 -> 63.36 | Helped |
| compute DCS similarity using channel tokens instead of patch tokens (ablation) | DiChaViT · CHAMMI | Avg score: 65.57 -> 69.68 | Helped |
| set DCS temperature to 0.1 rather than ≈ 0 (ablation) | DiChaViT · So2Sat | accuracy: 62.51 -> 63.36 | Helped |
| set DCS temperature to 0.1 rather than 0.2 (ablation) | DiChaViT · So2Sat | accuracy: 61.92 -> 63.36 | Helped |
| add channel tokens (ablation) | DiChaViT · JUMP-CP | accuracy: +8.0% | Helped |
| use orthogonal initialization (ablation) | DiChaViT · JUMP-CP and CHAMMI | accuracy: +1.0% | Helped |
| use a value of λ_CDL that is too large (ablation) | DiChaViT · So2Sat and CHAMMI | accuracy | Hurt |
Surprising findings
- Always sampling the least similar channels (temperature near zero) performs worse than moderate temperature sampling, so maximal dissimilarity is not optimal.
- Patch-token-based similarity for DCS underperforms channel-token-based similarity.
Limitations
- Cannot handle novel channels unseen during training.
- Requires extra hyperparameter tuning and compute.
- Evaluated only on three datasets with a small ViT-S backbone.
Experimental setup
Datasets: CHAMMI JUMP-CP So2Sat
Benchmarks: CHAMMI JUMP-CP So2Sat
Metrics: top-1 accuracy macro F1-score (1-NN, CHAMMI)
Baselines: ChannelViT ChAda-ViT DepthwiseViT TemplateMixingViT HyperNetViT
Builds on: ChannelViT Hierarchical Channel Sampling ProxyNCA++ DINOv2 ChAda-ViT
Closest papers by meaning
- Channel Vision Transformers: An Image Is Worth 1 x 16 x 16 Words (ICLR 2024 · similarity 0.91)
- DC-ViT: Modulating Spatial and Channel Interactions for Multi-Channel Images (NeurIPS 2026 · similarity 0.90)
- ChA-MAEViT: Unifying Channel-Aware Masked Autoencoders and Multi-Channel Vision Transformers for Improved Cross-Channel Learning (NeurIPS 2025 · similarity 0.89)
- ChAda-ViT : Channel Adaptive Attention for Joint Representation Learning of Heterogeneous Microscopy Images (CVPR 2024 · similarity 0.87)
- Linearly Decomposing and Recomposing Vision Transformers for Diverse-Scale Models (NeurIPS 2024 · similarity 0.85)
- Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.