Research map / Label-efficient and robust learning / Test-time adaptation

"What is Different Between These Datasets?" A Framework for Explaining Data Distribution Shifts

NeurIPS 2026 · Label-efficient and robust learning › Test-time adaptation · approach: Robust optimization for distribution shift · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an interpretable, model-agnostic framework for explaining differences between two same-domain datasets using influential examples over Rashomon-based intrinsic feature importances, prototype neighbourhood comparisons, and prototype summarization across tabular, time-series, audio, and image data.

Problem

Existing distribution-shift methods only detect whether shift occurred or summarize it with opaque statistics (e.g., Wasserstein, KL), leaving humans without interpretable explanations of how two same-domain datasets differ.

Approach

Combines Rashomon Importance Distribution feature importances with influence functions on a dataset discriminator to find examples whose removal aligns feature importances; adds prototype neighbourhood metrics (NSPD/NSDD) with partial-prototype feature selection and a ProtoPNet-style prototype summarization loss for comparing datasets in a shared latent space.

Key results

• Removing 20 influential examples (about 1% of the female subset) from Adult female data brings its GiFIMs closer to the male dataset's GiFIMs.
• Kendall-Tau similarity of influential examples across bootstraps is about 0.9975 for Adult and HELOC.
• On a mixture-of-Gaussians mean-shift example, the authors' prototype explanation remains coherent when cluster proportions change, while the optimal-transport baseline becomes uninterpretable.

Surprising findings

• Removing more examples beyond the positively influential ones reduces alignment, so the alignment benefit plateaus or reverses.

Limitations

• Influence approximations are validated mainly for logistic regression discriminators; correlation may be weaker for nonlinear models.
• Requires binarized features for GOSDT/RID, and the dataset-level alignment depends on tree-based task models.
• Prototype explanations depend on the choice of prototype count and feature subset (faithfulness-interpretability tradeoff).

Experimental setup

Datasets: Adult HELOC Stanford PPG dataset NeuroKit2 synthetic PPG RAVDESS Coqui TTS generated audio Office-Home mammography

Benchmarks: Adult HELOC

Metrics: feature importance alignment (% error reduction) theoretical vs empirical influence Kendall-Tau ranking similarity random triplet accuracy global permutation accuracy NSPD NSDD AUCC

Baselines: optimal transport dataset explanation (Mean-shift OT explanation)

Builds on: Rashomon Importance Distribution TreeFARMS GOSDT ProtoPNet PaCMAP influence functions

Closest papers by meaning

• A Fine-Grained Analysis on Distribution Shift (ICLR 2022 · similarity 0.89)
• Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift (NeurIPS 2019 · similarity 0.89)
• On the Need for a Language Describing Distribution Shifts: Illustrations on Tabular Datasets (NeurIPS 2023 · similarity 0.88)
• Estimating and Explaining Model Performance When Both Covariates and Labels Shift (NeurIPS 2022 · similarity 0.86)
• Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time (NeurIPS 2022 · similarity 0.86)
• quanda: An Interpretability Toolkit for Training Data Attribution Evaluation (NeurIPS 2026 · similarity 0.84)