Research map / Vision-language and multimodal / Multimodal LLMs
CerebraGloss: Instruction-Tuning a Large Vision-Language Model for Fine-Grained Clinical EEG Interpretation
Wei Gu, Luo Tianming, Qiran Zhang, Mohan Ye, Xiao Shen, Wenxin Chen, Yunhuan Li, Yichen Zhang, Jing Hong, Bao-liang Lu, Wei-Long Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Instruction-tunes a large vision-language model on EEG visualizations paired with fine-grained expert-style annotations to enable holistic clinical EEG interpretation.
Problem
Clinical EEG interpretation is laborious and subjective, and existing computational models only handle narrow classification tasks; LVLMs need datasets pairing EEG visualizations with fine-grained expert annotations, which are scarce.
Approach
Introduces a dataset pairing EEG visualizations with fine-grained, expert-level annotations and instruction-tunes an LVLM on it for holistic EEG interpretation rather than single-label classification.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| instruction-tune Qwen2.5-VL-3B on EEG data (CerebraGloss) | Qwen2.5-VL-3B · TUSZ | balanced accuracy: 55.02 -> 79.21 | Improved |
| instruction-tune on EEG data versus GPT-5 | CerebraGloss-3B · CerebraGloss-Bench MCQ | accuracy: 70.00 -> 80.00 | Improved |
| use CerebraGloss against specialized large EEG model on sleep staging | CerebraGloss-3B · HMC | balanced accuracy: 69.97 -> 62.02 | Hurt |
| add Stage 1 alignment stopped early at 0.05 epochs (ablation) | CerebraGloss · CerebraGloss-Bench | MCQ/Description/QA: MCQ 78.89 -> 80.00 | Improved |
| train Stage 1 longer (0.2 epochs) (ablation) | CerebraGloss · CerebraGloss-Bench MCQ | accuracy: 80.00 -> 74.44 | Hurt |
| add Stage 1 alignment (ablation) | CerebraGloss · TUSZ and HMC | balanced accuracy: TUSZ 79.68 vs 79.21 | No clear effect |
| add 94K LLM-generated instruction data (ablation) | CerebraGloss · CerebraGloss-Bench Description | ROUGE-1: 9.02 -> 44.19 | Improved |
| add 50K rule-based captions to Stage 2 (ablation) | CerebraGloss · CerebraGloss-Bench Description | ROUGE-1: Description 51.09 -> 44.19; MCQ 78.89 -> 80.00 | Mixed |
| scale base model from 3B to 7B (ablation) | CerebraGloss · TUSZ | balanced accuracy: 79.21 -> 80.21 | Improved |
| add general-domain data to mitigate forgetting | CerebraGloss · MMBench | accuracy | No clear effect |
Closest papers by meaning
- Neural Signals Generate Clinical Notes in the Wild (NeurIPS 2026 · similarity 0.90)
- NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces (NeurIPS 2026 · similarity 0.89)
- What do EEG Foundation Models Capture from Human Brain Signals? (NeurIPS 2026 · similarity 0.89)
- CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding (NeurIPS 2025 · similarity 0.88)
- CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding (ICLR 2025 · similarity 0.88)
- EEG Benchmarking Needs a Task Specification Layer: NeuroDoc for Rulebook-Guided, Executable Benchmark Construction (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
