Research map / Learning theory and algorithms / Prediction and decision losses
An engine not a camera: Measuring performative power of online search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measures the performative power of Google Search and Bing by running randomized experiments through a Chrome browser extension that reranks or hides search results and records clicks from ~85 participants.
Problem
Existing antitrust tools struggle to measure the economic power of digital platforms; the paper asks how much a search engine can steer web traffic by rearranging results, using the Google Shopping case as a case study.
Approach
Operationalize performative power (from performative prediction) via a browser extension that emulates algorithmic updates through randomized arrangements of search results, giving a causal, experimental lower bound on performative power via the performativity gap and algorithmic distortion factor.
Key results
- Down-ranking the top Google result by one position reduces its CTR from 43% to 24% (gap -0.19, algorithmic distortion 0.44).
- Combining box/Ads hiding with a swap yields a 66% algorithmic distortion for the first result on Google queries where Ads/boxes appear.
- Analysis of 45,625 clicks from 85 participants over five months (Sept 2023 to Jan 2024) with 98.9% on Google.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Swap the first and second generic search results, down-ranking c1 by one position | Google Search · Google Search queries | CTR of c1: 43% to 24%; -0.19 | Hurt |
| Swap the first and third generic search results, down-ranking c1 by two positions | Google Search · Google Search queries | Performativity gap for c1: -0.27 | Hurt |
| Swap the second and third generic search results, down-ranking c2 by one position | Google Search · Google Search queries | Algorithmic distortion for c2: 0.39 | Hurt |
| Swap the first and second generic search results, down-ranking c1 by one position | Bing Search · Bing Search queries | Click-through rate of c1 | Hurt |
| Show Shopping boxes on the search page | Google Search · Queries where Shopping boxes are present | Share of clicks absorbed by boxes: 22.4% | Hurt |
| Show Ads on the search page | Google Search · Queries where Ads are present | Share of clicks absorbed by Ads: close to 30% | Hurt |
| Hide Ads and Shopping boxes while swapping results 1–2 | Google Search · Queries with Shopping boxes present | Algorithmic distortion for c1: 25%; 66% | Hurt |
| Hide Shopping boxes while swapping results 1–2 | Google Search · Queries with Shopping boxes present | Algorithmic distortion for c1: 53% | Hurt |
Surprising findings
- Bing shows an even larger position effect than Google, though with much wider confidence intervals due to few queries.
- Shopping boxes and Ads absorb a large share of clicks, and combining their removal with down-ranking has a larger effect than either alone.
Limitations
- Convenience sample of about 85 trusted participants limits external validity.
- Bing results rest on a small sample (about 600 queries) and are less reliable.
- Counterfactuals were kept conservative to avoid user-experience degradation, so the estimates are lower bounds.
Experimental setup
Datasets: Powermeter click data from Google Search and Bing users
Metrics: click-through rate performativity gap algorithmic distortion factor
Builds on: performative prediction performative power SUTVA
Closest papers by meaning
- EcoGEO: Trajectory-Aware Evidence Ecosystems for Web-Enabled LLM Search Agents (NeurIPS 2026 · similarity 0.84)
- Promoting Sustainable Web Agents: Benchmarking and Estimating Energy Consumption Through Empirical and Theoretical Analysis (AAAI 2026 · similarity 0.81)
- On Online Experimentation without Device Identifiers (ICML 2024 · similarity 0.81)
- On the Convergence of No-Regret Dynamics in Information Retrieval Games with Proportional Ranking Functions (ICLR 2025 · similarity 0.81)
- Causal Inference out of Control: Estimating Performativity without Treatment Randomization (ICML 2024 · similarity 0.80)
- Mecha-nudges for Machines (NeurIPS 2026 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.