Research map / Probabilistic and scientific modeling / Causal discovery
Score-informed Neural Operator for Enhancing Ordering-based Causal Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SciNO, a functional diffusion model using a Fourier neural operator with learnable time encoding and real/imag spectral splitting, to stably estimate Hessian diagonals for score-matching ordering-based causal discovery (DiffAN, CaPS), plus a probabilistic control method that steers LLM causal ordering with SciNO evidence.
Problem
Score-matching ordering-based causal discovery under ANMs needs accurate Hessian diagonals of log-densities; Stein estimators are costly and memory-hungry, and diffusion-based second derivatives are unstable, degrading causal ordering in high dimensions.
Approach
Model the score with a time-conditioned Fourier neural operator in Sobolev/Hilbert space, replacing positional time embeddings with a learnable time encoding and splitting spectral signals into real and imaginary parts, to stabilize higher-order derivatives; additionally fuse SciNO variance-based evidence (rank or confidence-interval statistics from deep ensembles) with autoregressive LLM priors f
Key results
- SciNO reduces order divergence relative to DiffAN by 42.7% on synthetic graphs and 31.5% on real-world datasets on average.
- On ER(d100), DiffAN OD drops from 117.0 to 86.6 with SciNO (LTE).
- LLM control with SciNO evidence reduces Llama-3.1-8B order divergence by about 64% on average on BNLearn datasets with more than 40 nodes (up to 79% on MAGIC-NIAB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace DiffAN’s score model with SciNO | DiffAN causal ordering · ER synthetic graphs, D=10, 30, 50, 100 | OD: 3.2 ± 1.99 -> 2.7 ± 1.19; 26.7 ± 8.1 -> 16.9 ± 6.12; 45.5 ± 7.51 -> 32.8 ± 6.55; 117.0 ± 12.78 -> 86.6 ± 12.99 | Helped |
| Replace DiffAN’s score model with SciNO | DiffAN causal ordering · Real and semi-synthetic nonlinear ANM datasets | OD: 3.3 ± 0.78 -> 1.9 ± 0.54; 5.7 ± 2.69 -> 5.6 ± 2.65; 8.8 ± 6.85 -> 4.1 ± 0.70; 21.8 ± 4.26 -> 12.8 ± 3.19; 12.0 ± 3.87 -> | Helped |
| Replace DiffAN’s score model with SciNO | DiffAN causal ordering · Sachs(d8) | SHD: 22.8 ± 4.56 -> 23.1 ± 3.86 | Hurt |
| Use probed SciNO in CaPS | CaPS causal ordering · BNLearn linear ANM datasets | OD: 36 -> 37.5±0.50; 25 -> 25.4±0.80; 41 -> 41.6±0.80; 49 -> 48.2±0.87 | Mixed |
| Use positional embedding instead of LTE (ablation) | SciNO within DiffAN · ER synthetic graphs, D=30, 50, 100 | OD: 16.9 ± 6.12 -> 19.0 ± 6.91; 32.8 ± 6.55 -> 41.4 ± 11.33; 86.6 ± 12.99 -> 100.7 ± 15.62 | Hurt |
| Use positional embedding instead of LTE (ablation) | SciNO within DiffAN · ER(d10) | OD: 2.7 ± 1.19 -> 2.5 ± 1.36 | Mixed |
| Add probabilistic SciNO control to Llama-3.1-8B (1-norm) | Llama-3.1-8B causal ordering · BNLearn datasets with more than 40 nodes | OD: 9 -> 3.2 ± 0.40; 32 -> 10.6 ± 0.49; 18 -> 9.9 ± 0.54; 80 -> 20.4 ± 0.92 | Helped |
| Use SciNO confidence-interval control instead of rank control (ablation) | Llama-3.1-8B (1-norm) · MAGIC-NIAB(d44), ECOLI70(d46), MAGIC-IRRI(d64), ARTH150(d107) | OD: 3.2 ± 0.40 -> 2.0 ± 0.00; 10.6 ± 0.49 -> 9.9 ± 1.97; 9.9 ± 0.54 -> 11.1 ± 0.83; 20.4 ± 0.92 -> 18.9 ± 1.14 | Mixed |
| Use SciNO evidence instead of DiffAN evidence for LLM control | Autoregressive causal ordering · High-dimensional datasets | OD reduction: 46% -> 64% | Helped |
| Use SciNO evidence instead of uncontrolled autoregressive ordering | Autoregressive causal ordering · Real-world datasets | OD reduction: 64%; up to 79% | Helped |
Surprising findings
- Time-conditioned FNO with positional embeddings gives only slight gains over MLP-based DiffAN on causal ordering despite strong generative performance.
Limitations
- Relies on the additive noise model assumption, whose identifiability is restricted to additive functional relations.
- SciNO is currently limited to continuous random variables.
Experimental setup
Datasets: Erdös-Rényi synthetic graphs (GP-RBF nonlinear ANM) Physics Sachs BNLearn MAGIC-NIAB BNLearn ECOLI70 BNLearn MAGIC-IRRI BNLearn ARTH150
Benchmarks: Erdös-Rényi synthetic graphs (d2-d100); Physics; Sachs; BNLearn MAGIC-NIAB, ECOLI70, MAGIC-IRRI, ARTH150
Metrics: Order Divergence; Structural Hamming Distance; Structural In
Baselines: DiffAN CaPS SCORE GPT-4o Llama-3.1-8B-Instruct
Builds on: DiffAN CaPS SCORE Hilbert Diffusion Model Fourier Neural Operator
Closest papers by meaning
- A Fixed-Point Approach for Causal Generative Modeling (ICML 2024 · similarity 0.88)
- Ordering-Based Causal Discovery for Linear and Nonlinear Relations (NeurIPS 2024 · similarity 0.87)
- Learning Structural Causal Models from Ordering: Identifiable Flow Models (AAAI 2025 · similarity 0.87)
- Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference (ICLR 2025 · similarity 0.86)
- Optimal Kernel Choice for Score Function-based Causal Discovery (ICML 2024 · similarity 0.86)
- Since Faithfulness Fails: The Performance Limits of Neural Causal Discovery (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.