Research map / Probabilistic and scientific modeling / Causal discovery
Causal Fine-Tuning under Latent Confounded Shift
Jialin Yu, Yuxiang Zhou, Haoxuan Li, Junchi Yu, Mengyue Yang, Yulan He, Nevin L. Zhang, Philip Torr, Ricardo Silva
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Causal Fine-Tuning to standard fine-tuning | BERT SFT · Amazon stop-word injection, OOD spurious 10% | F1: 49.33 -> 56.40 | Improved |
| add Causal Fine-Tuning to standard fine-tuning | BERT SFT · data-source injection, OOD spurious 10% | F1: 37.78 -> 49.22 | Improved |
| add Causal Fine-Tuning to standard fine-tuning | BERT SFT · in-distribution test spurious 90% | F1: 92.39 -> 92.37 (Amazon); 91.39 -> 90.59 (data source) | No clear effect |
| CFT versus SWA under mild shift | BERT · Amazon stop-word injection, OOD spurious 10% | F1: SWA 59.75 vs CFT 56.40 | Hurt |
| CFT versus SWA under 8x scaled spurious noise | BERT · Amazon stop-word injection 8x noise, OOD spurious 10% | F1: 33.05 -> 33.11 | Improved |
| condition on Phi without causal adjustment (CFT-N) (ablation) | CFT · data-source injection, OOD spurious 10% | F1: 49.22 -> 15.05 | Hurt |
| add causal adjustment over Phi (CFT versus CFT-N) (ablation) | CFT · Amazon stop-word injection, OOD spurious 10% | F1: 48.00 -> 56.40 | Improved |
| use only causal representation C to predict (CFT-C) (ablation) | CFT · data-source injection, OOD spurious 10% | F1: 49.22 -> 42.25 | Hurt |
| use only low-level features Phi to predict (CFT-Phi) (ablation) | CFT · Amazon stop-word injection, OOD spurious 10% | F1: 56.40 -> 12.40 | Hurt |
| replace paired views (R0,R1) with identical views (R1,R1) (ablation) | CFT · data-source injection, OOD spurious 10% | F1: 49.22 -> 37.24 | Hurt |
| interpolate pretrained and fine-tuned weights (WISE) over SFT | BERT SFT · data-source injection, OOD spurious 10% | F1: 37.78 -> 31.83 | Hurt |
| add more training data (ablation) | CFT vs SFT · data-source injection | F1 | No clear effect |
Closest papers by meaning
- Identifying General Mechanism Shifts in Linear Causal Representations (NeurIPS 2024 · similarity 0.84)
- Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviors (ICML 2025 · similarity 0.84)
- Causal Representation Learning via Counterfactual Intervention (AAAI 2024 · similarity 0.84)
- Disentangled Representation Learning in Non-Markovian Causal Systems (NeurIPS 2024 · similarity 0.84)
- A Neural Framework for Generalized Causal Sensitivity Analysis (ICLR 2024 · similarity 0.84)
- Can Large Language Models Infer Causation from Correlation? (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
