Research map / Label-efficient and robust learning / Recognition and long-tail

Quilt: Robust Data Segment Selection against Concept Drifts

AAAI 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Prototype and concept dictionary models · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Quilt, a data-centric framework that selects historical data segments for training under concept drift, using gradient-based disparity and gain scores to discard drifted segments and keep useful ones.

Problem

Concept drift in data streams, where the joint distribution P(X,y) changes and degrades model accuracy; existing drift adaptation methods are model-centric and discard drifted historical data, while data subset selection methods assume only virtual drift (P(y|X) unchanged).

Approach

Extend gradient-based data subset selection (GLISTER/GRAD-MATCH-style) to concept drift by scoring each previous data segment with a disparity score (gradient distance to the current validation gradient, which proxies label change) and a gain score (inner product of average gradients), keeping segments with positive gain and low disparity.

Key results

• Quilt achieves the highest accuracy on SEA (.888), Random RBF (.833), Sine (.936), Electricity (.728) and Spam (.974) among compared methods in Table 2.
• On Random RBF, Quilt is 5.1x faster than the no-selection variant while using only 39.4% of data segments, with accuracy .833 vs .822.
• Segment selection recall against an exhaustive Best Segments oracle is 0.96-1.00 on the four synthetic datasets, with precision 0.75-0.98.

Measured effects reported in the paper

Surprising findings

• Removing the gain score gives slightly higher accuracy than full Quilt on Sine (.941 vs .936) while being much slower (0.4x speedup), so the gain score mainly helps efficiency.
• Full Data performs very poorly on Sine (.449), below Current Segment (.899), showing that naively using all historical data can hurt under drift.

Limitations

• Uses only a simple one-hidden-layer neural network; the authors note results with more complex models are untested.
• Disparity threshold requires Bayesian optimization per segment, adding tuning overhead.
• Real datasets' drift locations are unknown and segments are set heuristically (e.g., yearly for Weather), and Quilt's accuracy on Weather is not the best (Full Data .800 vs .796).

Experimental setup

Datasets: SEA Hyperplane Random RBF Sine Electricity NOAA Weather Spam Usenet1 Usenet2

Benchmarks: SEA Hyperplane Random RBF Sine Electricity NOAA Weather Spam Usenet1 Usenet2

Metrics: accuracy F1 score runtime (sec) precision recall

Baselines: Full Data Current Segment HAT ARF Learn++.NSE SEGA CVDTE GLISTER GRAD-MATCH

Builds on: GLISTER GRAD-MATCH CVDTE SEGA

Closest papers by meaning

• RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budget (NeurIPS 2025 · similarity 0.86)
• Early Concept Drift Detection via Prediction Uncertainty (AAAI 2025 · similarity 0.84)
• Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data (NeurIPS 2024 · similarity 0.84)
• A Bayesian Model Selection Criterion for Selecting Pretraining Checkpoints (ICML 2025 · similarity 0.84)
• A CLIP-Powered Framework for Robust and Generalizable Data Selection (ICLR 2025 · similarity 0.83)
• Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws (ICLR 2025 · similarity 0.83)