Research map / Federated learning and privacy / Privacy attacks and secure learning

Data Acquisition via Experimental Design for Data Markets

NeurIPS 2024 · Federated learning and privacy › Privacy attacks and secure learning · approach: Encryption-based private learning · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DAVED, a validation-free, federated data acquisition method for data markets that selects seller data via linear experimental design (V-optimal, Frank-Wolfe) to minimize the buyer's unlabeled test prediction error.

Problem

Data buyers in data markets must choose the most useful seller data points for their test queries under a budget, but validation-based data valuation overfits, needs labeled validation data, and does not scale to large seller corpora.

Approach

Replace validation-based proxy losses with a label-free V-optimal experimental design objective computed from covariates only (Fisher information of seller data vs. buyer test points), optimized by a federated Frank-Wolfe herding procedure where each seller computes its own gradient and communicates O(d) updates.

Key results

• On Gaussian 1K seller points, DAVED multi-step achieves test MSE 0.37 versus 1.24 for Data OOB and 1.38 for random selection.
• On DrugLib, DAVED single-step attains MSE 9.0 (best) versus 10.0 for Data OOB; on Fitzpatrick17K, DAVED multi-step reaches 0.67 vs 1.35 for Data OOB.
• On MIMIC 35K seller points, DAVED multi-step gets 171.4 MSE, while Data OOB gets 215.6 and random 283.7.

Measured effects reported in the paper

Surprising findings

• Validation-based data valuation methods can underperform a random selection baseline, and a theorem shows validation-based selection can be as bad as training only on the validation data (error scaling d/n_val).
• Data OOB, which also avoids a validation set, is the second-best method on most benchmarks.

Limitations

• Communicates at every optimization step; local steps (FedAvg/Scaffold-style) not yet integrated.
• Differential privacy guarantees are not provided and are left to future work.
• Assumes the conditional label distribution is identical across seller and buyer data and uses a linear approximation (eNTK or embeddings), and the Frank-Wolfe guarantee assumes bounded curvature.

Experimental setup

Datasets: Gaussian synthetic MIMIC-III RSNA Pediatric Bone Age Fitzpatrick17K DrugLib

Benchmarks: Gaussian synthetic MIMIC-III RSNA Pediatric Bone Age Fitzpatrick17K DrugLib

Metrics: test mean squared error runtime

Baselines: Data Shapley Leave One Out Influence DVRL LAVA KNN Shapley Data OOB Random baseline

Builds on: V-optimal experimental design Data Shapley neural tangent kernel Frank-Wolfe

Closest papers by meaning

• Domain-wise Data Acquisition to Improve Performance under Distribution Shift (ICML 2024 · similarity 0.83)
• Multi-Session Budget Optimization for Forward Auction-based Federated Learning (ICML 2025 · similarity 0.82)
• An Instrumental Value for Data Production and its Application to Data Pricing (ICML 2025 · similarity 0.82)
• Reputation-aware Revenue Allocation for Auction-based Federated Learning (AAAI 2025 · similarity 0.82)
• Fairshare Data Pricing via Data Valuation for Large Language Models (NeurIPS 2025 · similarity 0.82)
• Non-Linear Pricing Restores Tractability for a Data Seller (NeurIPS 2026 · similarity 0.81)