AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

The Overthinking Predicament: When Reasoning Hurts Ranking

Xuan Lu, Haohang Huang, Rui Meng, Yaohui Jin, Wenjun Zeng, Xiaoyu Shen

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

The paper examines whether explicit chain-of-thought reasoning helps LLM-based document rerankers, finding that it can hurt ranking quality (the "overthinking" predicament).

Problem

Document reranking in information retrieval; recent work adds explicit chain-of-thought reasoning to LLM rerankers, but its effectiveness for ranking is underexplored.

Approach

Systematic study of explicit CoT reasoning in LLM-based rerankers, showing that reasoning can degrade ranking quality rather than improve it. Abstract is truncated, so details of the proposed method are not stated.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add chain-of-thought reasoning to pointwise reranker (4B) (ablation)Direct-Point-4B · BRIGHT (original query)NDCG@10: 25.5 -> 16.5Hurt
add chain-of-thought reasoning to pointwise reranker (8B) (ablation)Direct-Point-8B · BRIGHT (original query)NDCG@10: 26.8 -> 20.7Hurt
add chain-of-thought reasoning to pointwise reranker (ablation)Direct-Point · BEIRNDCG@10: 45.4 -> 40.1 (4B); 43.7 -> 39.3 (8B)Hurt
add reasoning to listwise reranker (SFT+GRPO, 8B) (ablation)Direct-List-8B · BRIGHT (original query)NDCG@10: 27.1 -> 25.9Hurt
add reasoning to listwise reranker (SFT+GRPO, 8B) (ablation)Direct-List-8B · BEIRNDCG@10: 41.8 -> 39.9Hurt
add reasoning to pointwise reranker (ablation)Direct-Point-8B · BEIR calibrationECE: 0.105 -> 0.151Hurt
add reasoning to pointwise reranker (ablation)Direct-Point-8B · BRIGHT-Biology class-conditionalTPR/TNR: TPR 31.1 -> 50.5; TNR 100.0 -> 98.1Mixed
add GRPO after SFTlistwise rerankers · BRIGHT (original query)NDCG@10: 24.9 -> 27.1 (Direct-List-8B)Improved
add reasoning to listwise reranker (ablation)Direct-List-8B (SFT) · training split (100 instances)NDCG@10: 80.41 -> 82.57 mean, std 2.1 -> 3.2Mixed
add reasoning to listwise reranker (4B GRPO) (ablation)Direct-List-4B · MS MARCO DL19NDCG@10: 73.77 -> 70.76Hurt
GRPO instead of SFT for Reason-List rationale length (ablation)Reason-List-8B · training splitrationale tokens: 397.7 -> 172.3Improved

Surprising findings

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy