Research map / Large language models / Instruction and fine-tuning
Truthful Aggregation of LLMs with an Application to Online Advertising
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MOSAIC, a strategyproof auction that aggregates advertisers' LLM preferences over candidate replies via importance-sampling reweighting, with Rochet payments plus an offset, and no LLM fine-tuning.
Problem
Aggregating conflicting advertiser preferences over LLM-generated replies while keeping the reply useful to the user, with advertisers able to misreport their preferences (including their LLMs) and no access to fine-tuning or weights.
Approach
Generate M candidate replies from a (context-aware) proposal LLM and resample one by softmax over advertiser rewards plus log-ratio to the reference LLM; this converges to the RLHF-style optimal KL-regularized distribution as M grows. Payments are Rochet payments from a convex log-sum-exp utility, plus an offset that ties each advertiser's utility to her social-welfare contribution.
Key results
- Context-aware MOSAIC with 4 candidate replies exceeds baseline MOSAIC's log probability with 20 candidates.
- With 20 candidates the context-aware mechanism almost matches the estimated log probability of sampling from the optimal distribution.
- Pearson correlation between advertiser utility and reward is 0.8 with offset payments versus 0.4 with standard Rochet payments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add advertiser context when generating candidate replies | MOSAIC · Synthetic advertising instances | log probability under the optimal distribution: four candidate replies with context-aware MOSAIC; 20 with baseline MOSAIC | Helped |
| Add advertiser context when generating candidate replies | MOSAIC · Synthetic advertising instances | advertiser rewards | Helped |
| Add advertiser context when generating candidate replies | MOSAIC · Synthetic advertising instances | auctioneer revenue | Helped |
| Generate candidate replies from the reference LLM instead of the context-aware LLM (ablation) | MOSAIC · Synthetic advertising instances | advertiser rewards | Hurt |
| Increase the number of candidate replies | Context-aware MOSAIC · Synthetic advertising instances | log probability under the optimal distribution | Helped |
| Add the payment offset | MOSAIC · Synthetic advertising instances | Pearson correlation between advertiser utility and reward: 0.4 -> 0.8 | Helped |
| Add the payment offset | MOSAIC · Synthetic advertising instances | advertiser expected utility | Helped |
Surprising findings
- Baseline MOSAIC fails to improve advertiser rewards within feasible sample sizes and may slightly decrease them.
Limitations
- Ex-post individual rationality is impossible alongside strategyproofness and convergence; only almost-IR and ex-ante IR (empirical) are established.
- Experiments use synthetic instances with only two advertisers.
- Context-aware proposal construction lacks a formal framework.
Experimental setup
Datasets: synthetic user queries and advertiser descriptions (50 queries x 25 seeds)
Benchmarks: synthetic online advertising instances (1,250 instances)
Metrics: log probability of returned reply under optimal distribution normalized advertiser reward auctioneer revenue advertiser utility correlation with reward
Baselines: baseline MOSAIC (reference LLM as proposal) VCG (not applicable) sequential token-level LLM auction
Builds on: RLHF KL-regularized objective Rochet payments VCG cyclic monotonicity
Closest papers by meaning
- LLM-Auction: Generative Auction towards LLM-Native Advertising (NeurIPS 2026 · similarity 0.89)
- Ad Auctions for LLMs via Retrieval Augmented Generation (NeurIPS 2024 · similarity 0.88)
- Mechanism Design for LLM Fine-tuning with Multiple Reward Models (NeurIPS 2025 · similarity 0.86)
- Beyond Last-Click: An Optimal Mechanism for Ad Attribution (NeurIPS 2025 · similarity 0.85)
- Fairshare Data Pricing via Data Valuation for Large Language Models (NeurIPS 2025 · similarity 0.85)
- Test-Time Compute Games (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.