Research map / Learning theory and algorithms / Kernels and regression theory
On the Hardness of Conditional Independence Testing In Practice
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reframes the Kernel-based Conditional Independence (KCI) test as nearly a special case of the Generalized Covariance Measure family and analyzes how conditional mean embedding regression errors drive Type-I error inflation and how choosing the conditioning kernel trades off power against false rejections.
Problem
Kernel conditional independence tests such as KCI are known to have poor Type-I error control in practice, and the theoretical impossibility result of Shah and Peters says no universally valid test has nontrivial power, which does not explain observed practical failures.
Approach
Shows that GCM/weighted GCM is nearly a special case of KCI with linear kernels on A and B and a weight kernel on C; identifies conditional mean embedding regression error as the root cause of false rejections; proposes selecting the conditioning kernel by maximizing an estimated signal-to-noise ratio, which boosts power but inflates Type-I error; and bounds the null distribution approximation via
Key results
- On the synthetic example with m=200 training samples, Type-I error varies noticeably with the squared C-kernel lengthscale, while with m=1000 it stays controlled across lengthscales.
- In a multi-dimensional C experiment, shared-coordinate regression and dependence yields Type-I error 0.21 versus 0.10 for separate coordinates (Type-II 0.0 vs 0.08).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use separate coordinates for the regressions and conditional dependence instead of shared coordinates | KCI · Synthetic CI, Type-I error | Type-I error: 0.21 -> 0.10 | Helped |
| Use separate coordinates for the regressions and conditional dependence instead of shared coordinates | KCI · Synthetic CI, Type-II error | Type-II error: 0.0 -> 0.08 | Hurt |
| Select the conditioning kernel to maximize approximate test power rather than use a regression kernel for dependence measurement (ablation) | KCI · Synthetic CI, alternative | test power | Helped |
| Select the conditioning kernel by power maximization using noisy regressions (ablation) | KCI · Synthetic CI, null | Type-I error | Hurt |
| Increase the regression training sample size from m=200 to m=1000 (ablation) | KCI · Synthetic CI, null | Type-I error | Helped |
| Use a conditioning kernel with lengthscale ℓ_C=∞ (the GCM choice) instead of an intermediate lengthscale matched to the dependence frequency (ablation) | KCI · Synthetic CI, alternative | test power | Hurt |
| Use the average residual covariance without a localizing conditioning kernel (GCM) (ablation) | GCM · Synthetic CI, alternative | dependence detection | Hurt |
| Use Gamma calibration rather than the wild bootstrap (ablation) | KCI · Conditional independence testing | Type-II error | Hurt |
Surprising findings
- Maximizing test power by tuning the conditioning kernel can substantially inflate Type-I error, creating a tension between power and null calibration.
- Gaussian-kernel MMD between nearby point masses is small while total variation is maximal, so mean embedding estimation can succeed where distribution-based smoothness assumptions fail.
Limitations
- Theory relies on simplifying assumptions (Gaussian residuals, linear kernels on A and B) for analytic results.
- Null calibration remains delicate since no permutation-based exact procedure is available when the conditioning kernel is tuned.
- Regression errors must shrink with n for the test to remain valid; fixed regression error causes Type-I inflation.
Experimental setup
Metrics: Type-I error Type-II error test power signal-to-noise ratio
Baselines: KCI SplitKCI GCM Weighted GCM CIRCE RBPT KCI-Power
Builds on: KCI Generalized Covariance Measure Weighted GCM CIRCE SplitKCI
Closest papers by meaning
- Kernel conditional tests from learning-theoretic bounds (NeurIPS 2025 · similarity 0.88)
- A Conditional Independence Test in the Presence of Discretization (ICLR 2025 · similarity 0.88)
- Conditional Diffusion Models Based Conditional Independence Testing (AAAI 2025 · similarity 0.88)
- A Sample Efficient Conditional Independence Test in the Presence of Discretization (ICML 2025 · similarity 0.87)
- On the sample complexity of conditional independence testing with Von Mises estimator with application to causal discovery (ICML 2024 · similarity 0.87)
- Testing Conditional Mean Independence Using Generative Neural Networks (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.