Research map / Large language models / Language, safety and interpretability
Fairshare Data Pricing via Data Valuation for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a game-theoretic LLM data market framework where sellers (annotators) price data at buyers' maximum willingness to pay, computed via data valuation scores, showing that "fairshare" pricing beats exploitative pricing in long-run buyer and seller welfare.
Problem
Exploitative pricing of human-annotated training data for LLMs drives high-quality sellers out of the market, degrading data quality and long-term model performance; there is no formal model of fair LLM data pricing.
Approach
Model the LLM data market as a non-cooperative Stackelberg game with information transparency, price each dataset at the buyers' maximum willingness to pay derived from data valuation scores (the "fairshare" price), and show theoretically that this sustains seller participation and maximizes buyer cumulative utility.
Key results
- Fairshare pricing yields the highest seller profit over time and sustains seller participation over 100 time periods compared with exploitative pricing, which causes sellers to exit.
- In the valuation ablation, Infl_IP offered the best cost-performance trade-off, and Spearman correlation between Infl_IP and oracle values was 0.54 on MathQA and 0.42 on PIQA.
- Theory: any exploitative pricing is optimal only over a finite horizon and becomes strictly suboptimal in the long run under the stated assumptions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use fairshare pricing instead of exploitative pricing | LLM data market · Long-term market dynamics | seller participation and data supply | Helped |
| use fairshare pricing instead of exploitative pricing | LLM data market · Long-term market dynamics | buyer cumulative utility | Helped |
| use fairshare pricing instead of reduced, random, or exploitative pricing | LLM data market · Seller welfare over time | seller profit | Helped |
| use fairshare pricing instead of lower-pricing methods | High-budget buyer · Long-term market dynamics | buyer utility | Helped |
| use fairshare pricing instead of lower-pricing methods | Low-budget buyer · Long-term market dynamics | buyer utility | Mixed |
| use BM25, Infl_IP, or DataInf valuation instead of non-learning-aware valuation (ablation) | Fairshare pricing framework · MathQA, MedQA, and PIQA | model performance | Helped |
| use Infl_IP valuation instead of constant, random, or BM25 valuation (ablation) | Fairshare pricing framework · MathQA, MedQA, and PIQA | cost-performance balance | Helped |
| estimate values with Infl_IP rather than the Oracle one-step training approach (ablation) | Data valuation · MathQA | Spearman correlation with Oracle: 0.54 | Mixed |
| estimate values with Infl_IP rather than the Oracle one-step training approach (ablation) | Data valuation · PIQA | Spearman correlation with Oracle: 0.42 | Mixed |
Surprising findings
- The low-budget buyer experiences reduced short-term utility under fairshare pricing, benefiting only in the long run.
Limitations
- Seller participation is modeled by an assumed function pi(p, p*) = p/p*, not estimated from real annotator behavior.
- Oracle correlation of data valuation methods is only moderate (Spearman 0.42-0.54), and simulations use single-seller-per-sample markets with synthetic prices and budgets.
- Assumes complete information transparency and a single trusted valuation source; incomplete-information settings are left to future work.
Experimental setup
Datasets: MathQA GSM8K MedQA PIQA
Benchmarks: MathQA MedQA PIQA
Metrics: buyer cumulative utility seller cumulative profit active seller count model performance per dollar Spearman correlation with oracle valuation
Baselines: Reduced pricing Random pricing Exploitative pricing Constant valuation Random valuation BM25
Builds on: Influence functions DataInf Stackelberg game BM25
Closest papers by meaning
- A Theory of Training Profit-Optimal LLMs (NeurIPS 2026 · similarity 0.88)
- Test-Time Compute Games (NeurIPS 2026 · similarity 0.87)
- Pricing and Competition for Generative AI (NeurIPS 2024 · similarity 0.86)
- FinBen: A Holistic Financial Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.85)
- Truthful Aggregation of LLMs with an Application to Online Advertising (NeurIPS 2025 · similarity 0.85)
- Convex Dataset Valuation for Post-Training (ICML 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.