Research map / Label-efficient and robust learning / Knowledge distillation
Cross-Modal Representational Knowledge Distillation for Enhanced Spike-informed LFP Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes cross-modal representational knowledge distillation from a frozen multi-session spike transformer teacher (MS-Spike) into single-session LFP transformer students, improving LFP-based behavior decoding in unsupervised and supervised settings.
Problem
LFP-based neural models underperform spike-based models for behavior decoding, especially under unsupervised/self-supervised training, because LFPs are population-level aggregate, redundant, and noisy signals.
Approach
Distill latent representations from a pretrained multi-session spike transformer (with session-specific spatial embeddings and MAE pretraining) into an LFP transformer student by aligning paired spike/LFP representations via cosine similarity, combined with an LFP autoencoding loss; extends to multi-session distillation.
Key results
- Distilled LFP models achieved average unsupervised R^2 of 0.71 vs 0.27 for MS-LFP and 0.24 for SS-LFP, and slightly exceeded the MS-Spike teacher (0.69).
- In supervised fine-tuning, Distilled LFP reached 0.80 average R^2 (MS-Distilled LFP 0.82), outperforming all LFP-only baselines though not spike-based models (~0.85).
- Distilled LFP with 10x fewer parameters still outperformed baselines; a model distilled on a single session generalized to unseen sessions without further distillation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill spike representations into an LFP model | LFP models · Behavior decoding (Monkeys I, C, Ch, M) | R²: 0.27 -> 0.71 (MS-LFP); 0.24 -> 0.71 (SS-LFP) | Helped |
| Distill spike representations into an LFP model | LFP models · Behavior decoding (Monkeys I, C, Ch, M) | R²: 0.69 -> 0.71 | Helped |
| Use a multimodal spike-and-LFP model instead of cross-modal distillation | LFP behavior decoding · Behavior decoding (Monkeys I, C, Ch, M) | R²: 0.71 vs. 0.33 | Hurt |
| Zero spike inputs at inference in the multimodal model | SS-MM · Behavior decoding (Monkeys I, C, Ch, M) | R²: 0.71 vs. 0.27 | Hurt |
| Extend distillation from one session to multiple sessions | Distilled LFP · Behavior decoding, supervised fine-tuning | R²: 0.80 -> 0.82 | Helped |
| Extend distillation from one session to multiple sessions | Distilled LFP · Behavior decoding, fully-supervised fine-tuning | R²: 0.81 -> 0.83 | Helped |
| Extend distillation from one session to multiple sessions | Distilled LFP · Behavior decoding, unsupervised fine-tuning | R²: 0.71 -> 0.72 | No clear effect |
| Use a convolutional rather than linear value-embedding layer (ablation) | Distilled LFP and supervised SS-LFP models · Behavior decoding | R² | Helped |
Surprising findings
- Distilled LFP models performed slightly better than their spike teacher MS-Spike in the unsupervised setting (0.71 vs 0.69).
- Distilled LFP models outperformed multimodal SS-MM models that use both spikes and LFP at inference in the unsupervised setting (0.33 for SS-MM).
Limitations
- Requires paired spike and LFP recordings for distillation.
- Spike-based models still outperform Distilled LFP models in supervised setting (~0.85 vs 0.80).
- Evaluated only on motor cortical datasets from monkeys; LFP power features used for some datasets.
Experimental setup
Datasets: Makin et al. dataset Flint et al. dataset Gallego-Carracedo et al. dataset Perich et al. dataset Churchland et al. dataset Chowdhury et al. dataset Ma et al. dataset NeuroTask
Benchmarks: behavior decoding R^2 on held-out motor cortical sessions representation retrieval (top-1, top-5, mean rank) CKA
Metrics: R^2 top-1 accuracy top-5 accuracy mean rank CKA
Baselines: NDT2 MS-LFP SS-LFP SS-MM SS-MM-ZS MS-Spike
Builds on: NDT2 masked autoencoding rotary positional embeddings flash attention AdamW
Closest papers by meaning
- BLEND: Behavior-guided Neural Population Dynamics Modeling via Privileged Knowledge Distillation (ICLR 2025 · similarity 0.87)
- Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networks (CVPR 2025 · similarity 0.86)
- Standing on the Shoulders of Giants: Rethinking EEG Foundation Model Pretraining via Multi-Teacher Distillation (NeurIPS 2026 · similarity 0.86)
- Towards a "Universal Translator" for Neural Dynamics at Single-Cell, Single-Spike Resolution (NeurIPS 2024 · similarity 0.86)
- A Scalable, Causal, and Energy Efficient Framework for Neural Decoding with Spiking Neural Networks (NeurIPS 2025 · similarity 0.86)
- Latent Diffusion for Neural Spiking Data (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.