Research map / Label-efficient and robust learning / OOD and anomaly detection
Identifying Predictions That Influence the Future: Detecting Performative Concept Drift in Data Streams
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Defines performative drift in data streams and proposes CB-PDD, a checkerboard-based intervention test that detects self-fulfilling performative drift, evaluated on synthetic and semi-synthetic streams.
Problem
Detecting performative concept drift (drift induced by a deployed model's own predictions) in data streams when its presence is not known beforehand, since standard drift detectors cannot distinguish performative from intrinsic drift.
Approach
CB-PDD forces a fraction of incoming instances to be predicted by a parameterized checkerboard pattern that alternates labels over trial periods, then compares density changes for correct vs. incorrect predictions with a Mann-Whitney U test to isolate performative effects from intrinsic drift.
Key results
- CB-PDD detects performative drift with near-perfect detection rates on Water Potability and Loan Status semi-synthetic datasets for all sigma except the smallest (0.0001), but fails on the imbalanced Credit Card dataset.
- False detection rates for CB-PDD remain between 0% and 20% across all synthetic experiments.
- With a Threshold Classifier, ADWIN reliably detects performative drift at all but the lowest sigma and outperforms CB-PDD, while DDM fails to detect performative drift in all scenarios.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase performative drift strength σ in self-defeating feedback loops | CB-PDD · Synthetic self-defeating feedback loops | detection rate: 0.04 -> 0.5 -> 1.0 | Helped |
| Increase performative drift strength σ when only Class 1 is performative | CB-PDD · Synthetic data, only Class 1 performative | detection rate: 0.1 -> 0.72 -> 0.94 -> 0.96 | Helped |
| Use feature split f=0.5 instead of f=1.0 (ablation) | CB-PDD paired with Threshold Classifier · Threshold Classifier pairing (m=0.25) | detection rate: 0.58 -> 1.0 | Helped |
| Use feature split f=0.5 instead of f=1.0 (ablation) | CB-PDD paired with Threshold Classifier · Threshold Classifier pairing (m=0.5) | detection rate: 0.04 -> 0.12 | Helped |
| Use feature split f=0.5 instead of f=1.0 (ablation) | CB-PDD paired with Threshold Classifier · Threshold Classifier pairing (m=0.71) | detection rate: 0.02 -> 0.0 | Hurt |
| Use feature split f=0.5 instead of f=1.0 (ablation) | CB-PDD paired with Threshold Classifier · Threshold Classifier pairing (m=0.99) | detection rate: 0.02 -> 0.0 | Hurt |
| Increase trial length τ (ablation) | CB-PDD · Synthetic streams with performative drift | detection rate | Helped |
| Decrease feature split f in the high ε setting (ablation) | CB-PDD · Synthetic streams with high ε | detection rate | Hurt |
| Assign a larger share of instances to the deployed predictive model (ablation) | CB-PDD paired with a predictive model · Synthetic streams with performative drift | detection rate | Hurt |
| Use ADWIN with a Threshold Classifier rather than CB-PDD | Performative drift detection · Synthetic streams with a Threshold Classifier | detection rate | Helped |
| Use DDM as the traditional drift detector | Performative drift detection · Synthetic streams across investigated scenarios | detection rate | Hurt |
| Apply CB-PDD to the Credit Card dataset with imputed performative drift | CB-PDD · Credit Card | detection rate | Hurt |
Surprising findings
- ADWIN outperforms CB-PDD when paired with a Threshold Classifier, and DDM never detects performative drift.
- Detection rate drops sharply when a deployed classifier is mixed in (m=0.5 gives 0.04 for f=1.0 with a Threshold Classifier), since the classifier saturates into a stable state.
Limitations
- Only binary classification with self-fulfilling feedback loops was evaluated; self-defeating results are only in the appendix.
- CB-PDD is an intervention test requiring deliberate misclassification, and is sensitive to class imbalance (fails on Credit Card) and to trial length relative to intrinsic drift.
- Not presented as an online algorithm; no real-world data with known performative drift exists.
Experimental setup
Datasets: Water Potability Loan Status Credit Card RandomRBF-style synthetic streams
Benchmarks: synthetic performative streams Water Potability Loan Status Credit Card
Metrics: detection rate false detection rate
Baselines: ADWIN DDM
Builds on: Performative Prediction (Perdomo et al.) RandomRBF data generator ADWIN DDM KSWIN
Closest papers by meaning
- Early Concept Drift Detection via Prediction Uncertainty (AAAI 2025 · similarity 0.86)
- RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budget (NeurIPS 2025 · similarity 0.85)
- The Stability of Online Algorithms in Performative Prediction (NeurIPS 2026 · similarity 0.85)
- Optimal Classification under Performative Distribution Shift (NeurIPS 2024 · similarity 0.85)
- Attention Drift: What Auto-Regressive Speculative Decoding Models Learn (NeurIPS 2026 · similarity 0.85)
- Contribution of task-irrelevant stimuli to drift of neural representations (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.