Research map / Reinforcement learning / Policy gradient and actor-critic
MetaRLEC: Meta-Reinforcement Learning for Discovery of Brain Effective Connectivity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MetaRLEC, an actor-critic method with a transformer encoder, BiLSTM pointer decoder, and an online meta-critic that learns from historical state-action pairs to discover brain effective connectivity networks from small, noisy fMRI data.
Problem
Discovering brain effective connectivity networks from high-noise, small-sample fMRI time series, where existing methods perform suboptimally.
Approach
Combines an actor-critic reinforcement learning framework (BIC-based reward) with an online meta-critic that meta-learns an additional loss during a single task, using a bi-level update (inner: meta-training, outer: online meta testing) instead of offline task-family meta-learning.
Key results
- On Sim1, MetaRLEC achieves F1 0.63±0.17 and SHD 2.28±1.03, the best among the 9 methods compared.
- On Sim4 (15 nodes), MetaRLEC attains F1 0.51±0.14 and the lowest SHD 13.44±3.41, while Two-Step reaches F1 0.51±0.08 with SHD 14.58.
- Ablation on Sim1 shows removing the meta-critic network lowers F1 from 0.63 to 0.44±0.15, the largest drop among the ablated components.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MetaRLEC instead of Two-Step | Brain EC discovery · Sim1 | F1: 0.62 -> 0.63 | Helped |
| Use MetaRLEC instead of Two-Step | Brain EC discovery · Sim2 | F1: 0.49 -> 0.57 | Helped |
| Use MetaRLEC instead of Two-Step | Brain EC discovery · Sim3 | F1: 0.55 -> 0.56 | Helped |
| Use MetaRLEC instead of Two-Step | Brain EC discovery · Sim4 | F1: 0.51 -> 0.51 | No clear effect |
| Add the transformer-based encoder (ablation) | MetaRLEC · Sim1 | F1: 0.55±0.13 -> 0.63 ± 0.17 | Helped |
| Add the BiLSTM-based decoder (ablation) | MetaRLEC · Sim1 | F1: 0.47±0.21 -> 0.63 ± 0.17 | Helped |
| Add the meta-critic network (ablation) | MetaRLEC · Sim1 | F1: 0.44±0.15 -> 0.63 ± 0.17 | Helped |
Surprising findings
- On the largest simulation (Sim4), Two-Step and MetaRLEC have near-identical F1 (0.51), and MetaRLEC's recall is no better than several baselines, so gains are mostly in precision and SHD.
Limitations
- The BIC reward encourages directed acyclic graphs, which limits discovery of cyclic dependencies between brain regions.
- Real-data evaluation lacks full ground truth and relies on partial prior knowledge from prior studies.
- Only small graphs (up to 15 nodes) are evaluated; scaling to large-scale brain networks is left for future work.
Experimental setup
Datasets: Smith et al. 2011 NetSim simulated fMRI resting-state fMRI medial temporal lobe (Shah et al. 2018)
Benchmarks: Smith NetSim simulated fMRI (Sim1-Sim4) real resting-state fMRI medial temporal lobe
Metrics: Precision Recall SHD F1
Baselines: Patel pwLiNGAM lsGC Two-Step EC-RGAN RL-EC CR-VAE DiffAN
Builds on: Transformer Bi-LSTM pointer network meta-reinforcement learning RL-EC
Closest papers by meaning
- Meta-Learning an In-Context Transformer Model of Human Higher Visual Cortex (NeurIPS 2025 · similarity 0.86)
- MindBridge: A Cross-Subject Brain Decoding Framework (CVPR 2024 · similarity 0.84)
- Meta-Dynamical State Space Models for Integrative Neural Data Analysis (ICLR 2025 · similarity 0.84)
- A Local-Ascending-Global Learning Strategy for Brain-Computer Interface (AAAI 2024 · similarity 0.84)
- Unsupervised Representation Learning of Brain Activity via Bridging Voxel Activity and Functional Connectivity (ICML 2024 · similarity 0.84)
- MoRE-Brain: Routed Mixture of Experts for Interpretable and Generalizable Cross-Subject fMRI Visual Decoding (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.