Research map / Reinforcement learning / Visual and representation RL

Amortized Active Causal Induction with Deep Reinforcement Learning

NeurIPS 2024 · Reinforcement learning › Visual and representation RL · approach: Scalable RL systems and environments · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CAASL, a transformer intervention-design policy trained with REDQ/SAC on simulated causal environments, using AVICI amortized graph posterior to reward correct adjacency entries, enabling likelihood-free, forward-pass intervention design.

Problem

Active intervention design for causal structure learning under a fixed budget is usually likelihood-based and requires intermediate expensive causal graph inference at each step, which is infeasible in likelihood-free settings like gene regulatory networks.

Approach

Amortize intervention design into a single permutation-equivariant alternating-attention transformer policy trained with off-policy RL (REDQ/SAC) on simulator rollouts, with reward given by the expected number of correct adjacency entries under a pretrained AVICI amortized graph posterior; test-time design needs only a forward pass.

Key results

• On synthetic in-distribution environments with budget 10, CAASL reaches returns near 76 versus about 72 for random, and outperforms DiffCBED and SS Finite.
• CAASL shows zero-shot generalization to graph, mechanism, noise, and intervention-type shifts, beating random in most OOD settings, though gains over random are small for heteroskedastic noise.
• On SERGIO, CAASL beats random after 5 acquisitions in-distribution and remains robust under OOD graph, noise, knockdown, and noisy-knockout shifts; zero-shot higher-dimensional generalization is only on par with random.

Measured effects reported in the paper

Surprising findings

• Zero-shot generalization to higher dimensionality (d up to 30) is not reliably better than random, especially on SERGIO where high dropout is likely the cause.
• The random baseline remains very competitive under heteroskedastic noise and intervention-type changes.

Limitations

• Performance gains vanish under high missingness (around 74% dropout) when dimensionality increases, so zero-shot generalization to larger d is limited.
• Trained and evaluated only on simulated environments (synthetic linear SCMs and SERGIO); no real experimental data.
• Reward depends on the pretrained AVICI posterior and the policy is trained only on do (knockout) interventions.

Experimental setup

Datasets: synthetic linear additive-noise causal models SERGIO simulated single-cell data (10X Chromium noise)

Benchmarks: synthetic linear SCM design environments (in-distribution and OOD) SERGIO single-cell gene regulatory network environments (in-distribution and OOD)

Metrics: returns (expected correct adjacency entries) SHD AUPRC Edge F1

Baselines: Random Observational DiffCBED SS Finite

Builds on: AVICI SAC REDQ SERGIO DiffCBED

Closest papers by meaning

• ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition (NeurIPS 2025 · similarity 0.86)
• Hierarchical Reinforcement Learning with Targeted Causal Interventions (ICML 2025 · similarity 0.86)
• Amortized Molecular Optimization via Group Relative Policy Optimization (NeurIPS 2026 · similarity 0.86)
• Amortized Network Intervention to Steer the Excitatory Point Processes (ICLR 2024 · similarity 0.85)
• Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learning (ICLR 2024 · similarity 0.85)
• Adaptive teachers for amortized samplers (ICLR 2025 · similarity 0.85)