Research map / Probabilistic and scientific modeling / Molecules and proteins
DeepDRK: Deep Dependency Regularized Knockoff for Feature Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeepDRK, a transformer-based deep knockoff generator trained adversarially with multiple swappers and a sliced-Wasserstein correlation penalty, plus a post-training dependency-regularized perturbation, to improve FDR-controlled feature selection on non-Gaussian data.
Problem
Deep Model-X knockoff generators struggle to satisfy the swap property at the sample level, leading to overfitting, high reconstructability/collinearity, and low power, especially with small samples and non-Gaussian data.
Approach
Formulate knockoff generation as a min-max game against multiple neural swappers (multi-swapper adversarial attacks with a swapper diversity penalty and REx variance term), add a sliced Wasserstein correlation regularizer to reduce dependence between X and knockoffs, and post-hoc mix the knockoff with a row-permuted copy of X to further cut reconstructability.
Key results
- At n=200 on the Gaussian mixture setup, DeepDRK achieves mean FDR 0.116 versus 0.390 (KnockoffGAN), 0.720 (sRMMD), 0.735 (Deep Knockoff), and 0.772 (DDLK), at the cost of lower power (0.791 vs ~0.97-1.0 for baselines).
- At n=2000 on the Gaussian mixture setup, DeepDRK reaches mean FDR 0.081 with power 0.973, while Deep Knockoff and sRMMD have FDR above 0.63.
- On the IBD case study, DeepDRK identifies 23 metabolites of which 19 are literature-referenced, the most among compared models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DeepDRK instead of KnockoffGAN | Gaussian mixture feature selection · Gaussian mixture, n=200 | FDR mean: 0.390 -> 0.116 | Helped |
| Use DeepDRK instead of KnockoffGAN | Gaussian mixture feature selection · Gaussian mixture, n=200 | Power mean: 0.971 -> 0.791 | Hurt |
| Use DeepDRK instead of KnockoffGAN | Gaussian mixture feature selection · Gaussian mixture, n=2000 | FDR mean: 0.118 -> 0.081 | Helped |
| Use DeepDRK instead of KnockoffGAN | Gaussian mixture feature selection · Gaussian mixture, n=2000 | Power mean: 0.973 -> 0.973 | No clear effect |
| Decrease the DRP perturbation weight αₙ (ablation) | DeepDRK · Five synthetic datasets, n=2000 | Power | Helped |
| Vary the DRP perturbation weight αₙ (ablation) | DeepDRK · Five synthetic datasets, n=2000 | FDR | Mixed |
Surprising findings
- Competing objectives: minimizing the swap loss and the dependency loss trade off during training, so Type I and Type II error cannot both be reduced at the sample level.
- Baseline deep knockoff models achieve high power but FDR well above the nominal 0.1 level in small-sample synthetic settings.
Limitations
- FDR exceeds the 0.1 threshold in some small-sample (n=200) settings.
- Perturbation weight alpha_n is set heuristically (0.5) and may not be optimal; no theory on swap property and power for the full method.
- Real-data case study has no ground truth and relies on literature-based qualitative evaluation.
Experimental setup
Datasets: synthetic Gaussian mixture Clayton and Joe copula synthetic data 10x Genomics scRNA-seq IBD metabolomics (Metabolomics Workbench)
Benchmarks: synthetic Gaussian mixture copula synthetic semi-synthetic scRNA-seq semi-synthetic IBD metabolomics IBD case study
Metrics: FDR power literature-supported metabolites identified
Baselines: Deep Knockoff DDLK KnockoffGAN sRMMD
Builds on: Model-X knockoffs DDLK Deep Knockoff DeepPINK Vision Transformer
Closest papers by meaning
- Balancing Feature Similarity and Label Variability for Optimal Size-Aware One-shot Subset Selection (ICML 2024 · similarity 0.82)
- Knockoffs Inference for Partially Linear Models with Automatic Structure Discovery (AAAI 2025 · similarity 0.82)
- ACCon: Angle-Compensated Contrastive Regularizer for Deep Regression (AAAI 2025 · similarity 0.82)
- Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples for Deep Learning Models (ICML 2025 · similarity 0.81)
- FreqDebias: Towards Generalizable Deepfake Detection via Consistency-Driven Frequency Debiasing (CVPR 2025 · similarity 0.81)
- A CLIP-Powered Framework for Robust and Generalizable Data Selection (ICLR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.