Research map / Label-efficient and robust learning / Recognition and long-tail
Towards Homogeneous Lexical Tone Decoding from Heterogeneous Intracranial Recordings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes H2DiLR, a two-stage VQ-based framework that disentangles shared (homogeneous) and private (heterogeneous) neural codes from multi-subject sEEG to decode Mandarin lexical tones across 407 syllables.
Problem
Decoding lexical tones from heterogeneous intracranial recordings (different subjects, electrode configurations, physiology) is hard; subject-specific models generalize poorly with scarce invasive data.
Approach
Use a vector-quantized autoencoder with a shared codebook (homogeneous tone-articulation codes across subjects) and per-subject private codebooks (heterogeneous codes), splitting tokens by nearest-code ranking, then train a lightweight transformer decoder on the frozen disentangled representations.
Key results
- H2DiLR achieves 43.67% average top-1 tone decoding accuracy vs 37.47% for the best heterogeneous baseline in the main table (about 12.2% average improvement claimed over subject-specific models).
- UPaNT-only homogeneous model reaches 40.37% average; H2D adds about 3.3 points.
- On CHB-MIT seizure prediction, H2DiLR reaches 93.97% accuracy vs 89.36% for NeuroBERT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add H2D disentanglement to UPaNT (ablation) | H2DiLR · Mandarin tone decoding | top-1 accuracy (%): 40.37 $\pm$2.11 -> 43.67 $\pm$1.78 | Helped |
| add H2D disentanglement to UPaNT (ablation) | H2DiLR · CHB-MIT seizure prediction | accuracy (%): 91.19 $\pm$2.34 -> 93.97 $\pm$2.16 | Helped |
| remove the shared codebook (ν=0) (ablation) | H2DiLR · Mandarin tone decoding | top-1 accuracy (%): 38.98 $\pm$1.67 -> 43.67 $\pm$1.78 | Hurt |
| vary partition ratio ν from 0.25 to 0.75 (ablation) | H2DiLR · Tone decoding, combined representations | accuracy: 42.78, 43.67, 41.31 | Mixed |
| set partition ratio ν to 1.0 (UPaNT only) (ablation) | H2DiLR · Tone decoding, combined representations | accuracy: 40.37 -> 43.67 | Hurt |
| reduce code dimension from 256 to 64 (ablation) | H2DiLR · Tone decoding (UPaNT+H2D) | accuracy: 43.67 -> 39.76 | Hurt |
| increase code dimension from 256 to 512 (ablation) | H2DiLR · Tone decoding (UPaNT+H2D) | accuracy: 43.67 -> 43.35 | No clear effect |
| reduce code size from 256 to 64 (ablation) | H2DiLR · Tone decoding (UPaNT+H2D) | accuracy: 43.67 -> 39.27 | Hurt |
| increase code size from 256 to 512 (ablation) | H2DiLR · Tone decoding (UPaNT+H2D) | accuracy: 43.67 -> 43.59 | No clear effect |
| deepen and widen the Transformer to Transformer-8-256 (ablation) | H2DiLR · Tone decoding (all four subjects) | top-1 accuracy (%): 43.67 $\pm$1.78 -> 45.82 $\pm$ 1.63 | Helped |
| scale both encoder and Transformer to larger configurations (ablation) | H2DiLR · Tone decoding (all four subjects) | top-1 accuracy (%): 43.67 $\pm$1.78 -> 45.95 $\pm$ 1.65 | Helped |
| increase training participant count from 2 to 3 (ablation) | H2DiLR · Tone decoding | top-1 accuracy (%): 46.25 $7$1.89 -> 48.16 $\pm$ 2.04 | Helped |
Surprising findings
- Better VQ reconstruction (lower MSE) does not necessarily yield better tone decoding accuracy.
- Simply aggregating multi-subject data via BIOT did not significantly beat heterogeneous subject-specific baselines.
Limitations
- Generalization to unseen new subjects is not explored.
- Learned neural codes lack interpretability mapping to brain regions.
- Only four participants could be recruited due to intracranial data acquisition constraints.
Experimental setup
Datasets: In-house sEEG tone decoding dataset (4 participants, 407 Mandarin syllables) CHB-MIT
Benchmarks: In-house sEEG tone decoding (407 syllables) CHB-MIT seizure prediction
Metrics: top-1 accuracy sensitivity false positive rate per hour MSE subject classification accuracy
Baselines: SPaRCNet FFCL TS-TCC CoST ST-Transformer NeuroBERT BIOT UPaNT
Builds on: VQ-VAE ViT BIOT UPaNT
Closest papers by meaning
- Du-IN: Discrete units-guided mask modeling for decoding speech from Intracranial Neural signals (NeurIPS 2024 · similarity 0.88)
- The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning (ICML 2025 · similarity 0.87)
- Neuro-Vision to Language: Enhancing Brain Recording-based Visual Reconstruction and Language Interaction (NeurIPS 2024 · similarity 0.86)
- Far from the Shallow: Brain-Predictive Reasoning Embedding through Residual Disentanglement (NeurIPS 2025 · similarity 0.86)
- MIRAGE: Adaptive Multimodal Gating for Whole-Brain fMRI Encoding (NeurIPS 2026 · similarity 0.86)
- Brain-tuning Improves Generalizability and Efficiency of Brain Alignment in Speech Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.