Research map / Probabilistic and scientific modeling / Treatment effects

Isolated Causal Effects of Natural Language

ICML 2025 · Probabilistic and scientific modeling › Treatment effects · approach: Latent-variable causal inference · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces a doubly robust estimation framework for isolated causal effects of language-encoded interventions, with omitted-variable-bias-based fidelity, overlap, and robustness-value diagnostics for approximations of non-focal language.

Problem

Estimating the causal effect of a focal language attribute while holding the non-focal text at a target distribution, since non-focal language is aliased with the intervention and approximating it can introduce omitted variable bias.

Approach

Defines the isolated effect via a target distribution P* over non-focal language, estimates it with a doubly robust (IPW plus outcome model) estimator using short non-focal representations, and bounds omitted variable bias through fidelity (sigma^2), overlap (nu^2), and a robustness value.

Key results

• On the SvT dataset, SenteCon-Empath yields an isolated effect estimate very close to the clinical trial ground truth and has among the highest robustness values, while all estimates have wide 95% CIs that include 0.
• On the Amazon semi-synthetic outcome, the isolated estimates move closer to ground truth as non-focal representation dimensionality increases (after about 5 dimensions), with robustness values generally increasing.
• SVD dimensionality reduction improves overlap and robustness value for RoBERTa and MPNet embeddings on SvT, moving their point estimates inside the true-effect confidence interval.

Measured effects reported in the paper

Surprising findings

• Masking key information such as medication type and weight-loss terms from the text reduced explanatory power much more than omitting relevant lexical categories, yet the OVB lower bound stayed positive.
• Simple lexicon SenteCon-Empath, not designed for the task, achieved robustness comparable to LLM prompting with extensive human design.

Limitations

• Assumes the codebook function a(X) is known exactly; estimation error in a(X) would add bias not analyzed.
• Assumes no external confounding beyond text, relying on randomized annotator assignment.
• Confidence intervals on the SvT real-world setting are wide, so point estimates alone cannot select the best representation.

Experimental setup

Datasets: Amazon product reviews (helpfulness votes, semi-synthetic outcome) Semaglutide vs. Tirzepatide Reddit posts (SvT) LIWC Empath

Benchmarks: Amazon semi-synthetic helpfulness (linear and nonlinear outcomes) SvT clinical trial true effect

Metrics: isolated effect estimate and 95% CI fidelity sigma^2 overlap nu^2 robustness value OVB lower bound

Baselines: naive natural effect estimator

Builds on: doubly robust estimation omitted variable bias sensitivity bounds for non-parametric models (Chernozhukov codebook framework for causal text inference SenteCon Dhawan et al. SvT weight-loss estimation

Closest papers by meaning

• End-To-End Causal Effect Estimation from Unstructured Natural Language Data (NeurIPS 2024 · similarity 0.88)
• Can Large Language Models Infer Causation from Correlation? (ICLR 2024 · similarity 0.85)
• Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviors (ICML 2025 · similarity 0.84)
• Proximal Causal Inference With Text Data (NeurIPS 2024 · similarity 0.84)
• Are Models Biased on Text without Gender-related Language? (ICLR 2024 · similarity 0.84)
• Transferring Causal Effects using Proxies (NeurIPS 2025 · similarity 0.84)