AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Disentangling Channel Semantics in Vision Transformers via Token Decorrelation and Composition-Aware Modulation

Daeun Kim, Hyejin Park, Hyesong Choi, Dongbo Min

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: Efficient and robust vision transformers · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a multi-channel Vision Transformer that disentangles channel semantics via token decorrelation and composition-aware modulation, building on memory-token approaches that augment the [CLS] token.

Problem

Multi-channel images have heterogeneous channel semantics over a shared spatial structure; existing multi-channel ViTs that augment [CLS] with memory tokens struggle to jointly capture globally shared structure and channel-specific variation under varying channel compositions.

Approach

Decorrelate tokens to separate channel-specific from shared information and use composition-aware modulation so representations adapt to which channels are present, rather than relying only on memory tokens appended to [CLS].

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add FTL to ChA-MAEViT (ablation)ChA-MAEViT · JUMP-CP (all channels)accuracy: 90.35 -> 91.58Improved
add FTL to ChA-MAEViT (ablation)ChA-MAEViT · JUMP-CP (partial channels)accuracy: 68.10 -> 68.23No clear effect
add CAM to ChA-MAEViT (ablation)ChA-MAEViT · JUMP-CP (all channels)accuracy: 90.35 -> 92.46Improved
add both FTL and CAM (ablation)ChA-MAEViT · JUMP-CP (all channels)accuracy: 90.35 -> 94.35Improved
condition CAM on FTL-refined CLS rather than raw CLS (ablation)CAM · JUMP-CP (all channels)accuracy: 93.18 -> 95.41Improved
condition CAM on CLS instead of external channel multi-hot vector (ablation)CAM · JUMP-CP (all channels)accuracy: 92.57 -> 95.41Improved
use asymmetric weight in FTL instead of equal weight (ablation)FTL · JUMP-CP (all channels)accuracy: 94.09 -> 95.41Improved
apply CAM in last two blocks instead of first two (ablation)CAM · JUMP-CP (all channels)accuracy: 92.18 -> 95.41Improved
add FTL+CAM to IC-ViT memIC-ViT mem · JUMP-CP (all channels)accuracy: 86.68 -> 87.83Improved
add FTL+CAM to DiChaViT memDiChaViT mem · JUMP-CP (all channels)accuracy: 64.83 -> 68.09Improved
use MuCa-ViT at ViT-Base scale on CHAMMIMuCa-ViT · CHAMMIavg score: 76.20 -> 76.72Improved
use MuCa-ViT versus ChA-MAEViT on So2Sat partial channelsMuCa-ViT · So2Sat (partial channels)accuracy: 49.83 -> 52.12Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Surprising findings

Experimental setup

Baselines: multi-channel Vision Transformers with memory tokens

Builds on: memory tokens ChannelViT

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy