Research map / Architectures and efficiency / Attention and transformers

LUNA: Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysis

NeurIPS 2025 · Architectures and efficiency › Attention and transformers · approach: Block-sparse attention kernels · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LUNA, a self-supervised EEG foundation model that uses learned-query cross-attention to map variable electrode layouts into a fixed latent space, then runs temporal attention on that latent, achieving linear-in-channel cost.

Problem

EEG corpora use heterogeneous electrode montages, so models do not transfer across datasets; existing joint spatio-temporal attention scales quadratically with channel count and patch count.

Approach

Learned queries with cross-attention compress channel features at each patch into Q latent tokens (linear in channel count), then patch-wise temporal transformers operate on the compact latent sequence, with 3D sinusoidal electrode position encodings and a query specialization loss; pre-trained with masked-patch reconstruction.

Key results

• LUNA-Huge reaches 0.921 AUROC on TUAR and 0.802 AUROC on TUSL, the best reported in the paper for those tasks.
• On TUAB, LUNA-Huge reaches 0.8957 AUROC, below LaBraM-Huge (0.9162) and CBraMod (0.9156).
• On SEED-V (unseen 62-channel montage), LUNA lags CBraMod by roughly 2-3 pp balanced accuracy; the paper reports about 300x fewer FLOPs than BIOT at high patch counts is not claimed, but FLOPs/memory scale favorably versus BIOT in the appendix tables.

Measured effects reported in the paper

Surprising findings

• On SEED-V, the unseen 62-channel montage, LUNA underperforms CBraMod and LaBraM, which the authors attribute to positional encoding constraints.
• Ablating the query specialization loss gives only a non-significant TUAR AUROC change (p=0.2136), and region-based attention is within seed variation.

Experimental setup

Datasets: TUEG Siena TUAB TUAR TUSL SEED-V

Benchmarks: TUAB TUAR TUSL SEED-V

Metrics: AUROC AUC-PR balanced accuracy Cohen's kappa weighted F1 GFLOPs memory

Baselines: LaBraM CBraMod EEGFormer BIOT FEMBA CEReBrO BrainBERT SPaRCNet

Builds on: Set Transformer Perceiver IO LaBraM CBraMod RoPE

Closest papers by meaning

• What do EEG Foundation Models Capture from Human Brain Signals? (NeurIPS 2026 · similarity 0.87)
• CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding (NeurIPS 2025 · similarity 0.86)
• BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals (NeurIPS 2025 · similarity 0.85)
• NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces (NeurIPS 2026 · similarity 0.85)
• B[FM]$^2$: Brain Foundation Model via Flow Matching with SplitUNet (NeurIPS 2026 · similarity 0.85)
• THD-BAR: Topology Hierarchical Derived Brain Autoregressive Modeling for EEG Generic Representations (NeurIPS 2025 · similarity 0.84)