Research map / Architectures and efficiency / Attention and transformers
ChAda-ViT : Channel Adaptive Attention for Joint Representation Learning of Heterogeneous Microscopy Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ChAda-ViT, a channel-adaptive ViT with inter-channel attention, padding/masking and channel embeddings, trained with DINO on the new IDRCell100k microscopy dataset to yield a joint embedding for heterogeneous channel counts.
Problem
Microscopy images vary in channel number, order and type across experiments, so standard ViTs (which encode channels independently or assume RGB) cannot produce a unified representation or model inter-channel relationships.
Approach
Patchify each channel separately, pad token sequences to the maximum channel count with attention masking, and add learnable channel embeddings plus shared positional embeddings so that a single ViT applies both intra-channel and inter-channel attention, producing a fixed-size embedding for any channel configuration.
Key results
- ChAda-ViT outperforms the one-channel ViT on 6 of 8 downstream tasks using CLS token only (R2 normalized 0-100 for BBBC021 and nuclear translocation).
- One Channel ViT performs better on BBBC048 classification and clustering, attributed to cell-cycle features being present within single channels.
- ChAda-ViT Moyen produces a 192-dimensional embedding regardless of channel count, versus 384-576 dimensions for One Channel ViT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add inter-channel attention alongside intra-channel attention | ChAda-ViT · Downstream tasks (6 of 8) | task performance | Helped |
| Use ChAda-ViT instead of One Channel ViT | BBBC048 · BBBC048 | accuracy | Hurt |
| Use inter-channel attention only, without intra-channel attention (ablation) | Inter-Channel only ViT · Biological image downstream tasks | representation quality | Hurt |
Surprising findings
- Inter-Channel-only ViT appears insufficient alone; combining inter- and intra-channel attention gives the best representations.
- The one-channel baseline wins on BBBC048 classification, where per-channel features already suffice.
Limitations
- Underperforms on tasks heavily reliant on intra-channel information (e.g., BBBC048).
- Processes many more tokens than a standard ViT-Tiny, increasing compute cost.
- Cross-dataset joint space shown only qualitatively with two datasets; label alignment not solved.
Experimental setup
Datasets: IDRCell100k BloodMNIST CyclOPS BBBC048 NF-kB nuclear translocation BBBC021 Bray
Benchmarks: BloodMNIST CyclOPS BBBC048 NF-kB nuclear translocation BBBC021
Metrics: accuracy R2 MSE MAE
Baselines: One Channel ViT (Microsnoop-style) Inter-Channel only ViT
Builds on: DINO Vision Transformer Microsnoop
Closest papers by meaning
- Channel Vision Transformers: An Image Is Worth 1 x 16 x 16 Words (ICLR 2024 · similarity 0.88)
- DC-ViT: Modulating Spatial and Channel Interactions for Multi-Channel Images (NeurIPS 2026 · similarity 0.87)
- Enhancing Feature Diversity Boosts Channel-Adaptive Vision Transformers (NeurIPS 2024 · similarity 0.87)
- Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology (CVPR 2024 · similarity 0.87)
- Convolution Goes Higher-Order: A Biologically Inspired Mechanism Empowers Image Classification (NeurIPS 2025 · similarity 0.85)
- Micro-Bench: A Microscopy Benchmark for Vision-Language Understanding (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.