atlas

Research map / Large language models / Retrieval and knowledge

Ad Auctions for LLMs via Retrieval Augmented Generation

NeurIPS 2024 · Large language models › Retrieval and knowledge · approach: RAG robustness and evaluation · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes segment auctions that allocate ads within LLM outputs via RAG-style probabilistic retrieval, with truthful pricing and a logarithmic social welfare objective, evaluated with GPT-4 simulations.

Problem

How to sell ad placements inside LLM textual outputs so that ad allocation is efficient, fair, incentive compatible and individually rational, while preserving output quality.

Approach

Treat RAG's retrieval probabilities as the allocation rule of an auction: bids reweight calibrated relevance, a Gumbel-perturbed second-price auction implements the randomized allocation, and each discourse segment gets its own auction (single or multi-ad). The single-ad rule is shown to maximize a logarithmic social welfare and to be truthful.

Key results

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Allow repeated ad selection across segments rather than requiring different adsSegment auction · Four-ad scenariosocial welfare: $.521$ -> $.660$Helped
Allow repeated ad selection across segments rather than requiring different adsSegment auction · Four-ad scenariorevenue: $.333$ -> $.371$Helped
Allocate three ads to the entire output at once instead of one ad per segmentSegment auction · Four-ad scenariorevenue: $.371$ -> $.238$Hurt
Allocate three ads to the entire output at once instead of one ad per segmentSegment auction · Output similarity, first three segmentscosine similarity: $.700$ -> $.715$Helped
Integrate selected ads into the output with the LLM instead of appending their textSegment auction · Output similarity, first three segmentscosine similarity: $.671$ -> $.700$Helped
Use relevance scores rather than disregarding relevanceSegment auction with replacement · Four-ad scenariosocial welfare: $.508$ -> $.660$Helped

Surprising findings

Limitations

Experimental setup

Datasets: hand-written advertiser descriptions for simulated scenarios

Benchmarks: simulated ad auction scenarios

Metrics: revenue social welfare relevance minimum social welfare embedding cosine similarity of output quality

Baselines: Naive I (concatenate ads at end) Naive II (ignores relevance) segment auction without replacement

Builds on: retrieval augmented generation Myerson's lemma Gumbel max trick Nash social welfare

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy