Research map / Large language models / Retrieval and knowledge
C-SEO Bench: Does Conversational SEO Work?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces C-SEO Bench, a multi-task, multi-domain, multi-adopter benchmark that tests white-hat conversational search engine optimization methods, finding most are ineffective or harmful while traditional SEO (retrieval position) gives much larger gains.
Problem
Existing C-SEO methods are tested on few domains and only with a single adopter, so it is unclear whether they improve LLM citation ranking across tasks and competitive settings.
Approach
Measure C-SEO effectiveness by citation-rank improvement across two tasks (product recommendation, question answering) and six domains, and model multiple concurrent adopters via an adoption-rate protocol with AUC as a summary metric; also compare against traditional SEO by in-context position.
Key results
- Most C-SEO methods yield near-zero average rank gain; only LLM guidance (retail +0.36, games +0.24) and content improvement (retail +0.18) are significant on gpt-4o-mini, and many methods significantly lower rankings (e.g., Statistics decreases in 19 of 24 settings).
- Best SEO (placing the document first in context) gives gains of 2.77 on retail and 0.87 on web QA for gpt-4o-mini, far larger than any C-SEO method.
- Average per-adopter gain decreases as adoption rate rises from 10% to 100%, indicating a congested, zero-sum dynamic.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Prepend an LLM-generated summary to the document | C-SEO methods on gpt-4o-mini · Retail | rank improvement: 0.36 | Helped |
| Prepend an LLM-generated summary to the document | C-SEO methods on gpt-4o-mini · Video Games | rank improvement: 0.24 | Helped |
| Apply Content Improvement rewriting | C-SEO methods on gpt-4o-mini · Retail | rank improvement: 0.18 | Helped |
| Place the target document first in the LLM context | Conversational search on gpt-4o-mini · All six domains | rank improvement: 2.77; 1.89; 1.60; 0.87; 0.70; 1.54 | Helped |
| Apply Statistics document transformation | C-SEO methods · Across evaluated settings | rank improvement | Hurt |
| Increase the adoption rate of the same C-SEO method | C-SEO adopters · Retail and Video Games | average rank improvement per adopter | Hurt |
| Apply existing C-SEO transformations | C-SEO methods on Haiku 3.5 · Product Recommendation | rank improvement | Hurt |
Surprising findings
- Many C-SEO methods significantly decrease document citation rank rather than increasing it.
- Traditional SEO (retrieval position) dominates content modification in effect size.
Limitations
- Only white-hat C-SEO methods are evaluated; black-hat attacks are out of scope.
- Experiments use only proprietary commercial models.
- Dataset is restricted to English-language content.
Experimental setup
Datasets: Amazon Shopping Queries Steam Games dataset Google Books API Natural Questions Brave Search API web pages Multi-News debate verification snippets dataset
Benchmarks: C-SEO Bench
Metrics: citation rank improvement AUC over adoption rate Wilcoxon signed-rank p-value
Baselines: Authoritative Statistics Citations Fluency Unique Words Technical Terms Simple Language Quotes
Builds on: Generative Engine Optimization Amazon Shopping Queries Multi-News Natural Questions
Closest papers by meaning
- SNS-Bench: Defining, Building, and Assessing Capabilities of Large Language Models in Social Networking Services (ICML 2025 · similarity 0.84)
- DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation (NeurIPS 2025 · similarity 0.83)
- Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers (ICLR 2025 · similarity 0.83)
- FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents (NeurIPS 2025 · similarity 0.83)
- LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs – No Silver Bullet for LC or RAG Routing (ICML 2025 · similarity 0.83)
- WideSearch: Benchmarking Agentic Broad Info-Seeking (ICLR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.