Research map / Large language models / Agents, code and math
Navigating Through Paper Flood: Advancing LLM-Based Paper Evaluation Through Domain-Aware Retrieval and Latent Reasoning
Wuqiang Zheng, Yiyan Xu, Xinyu Lin, Chongming Gao, Wenjie Wang, Fuli Feng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add domain-aware paper retrieval (ablation) | PaperEval · NAID | NDCG@10: 0.9432 -> 0.9589 | Improved |
| add domain-aware paper retrieval (ablation) | PaperEval · ICLR | Spearman: 0.3545 -> 0.3276 | Hurt |
| add latent reasoning (ablation) | PaperEval · NAID | NDCG@10: 0.9449 -> 0.9589 | Improved |
| add latent reasoning (ablation) | PaperEval · NAID | Spearman: 0.4968 -> 0.4953 | No clear effect |
| add latent reasoning (ablation) | PaperEval · ICLR | Spearman: 0.3479 -> 0.3276 | Hurt |
| add progressive ranking optimization instead of MSE (ablation) | PaperEval · NAID | NDCG@10: 0.9585 -> 0.9589 | No clear effect |
| add progressive ranking optimization instead of MSE (ablation) | PaperEval · ICLR | NDCG@10: 0.7166 -> 0.7784 | Improved |
| list-wise instead of pair-wise ranking loss (ablation) | PaperEval · NAID | NDCG@10: 0.9296 -> 0.9589 | Improved |
| ranking loss instead of KL distribution-similarity loss (ablation) | PaperEval · ICLR | NDCG@10: 0.7559 -> 0.7784 | Improved |
| increase number of reasoning steps m beyond moderate value (ablation) | PaperEval · NAID | N@10 / Spearman | Hurt |
| increase retrieved reference papers k too far (ablation) | PaperEval · NAID | NDCG@10 | Hurt |
Closest papers by meaning
- Not Search, But Scan: Benchmarking MLLMs on Scan-Oriented Academic Paper Reasoning (ICLR 2026 · similarity 0.86)
- GraphEval: A Lightweight Graph-Based LLM Framework for Idea Evaluation (ICLR 2025 · similarity 0.84)
- From Replication to Redesign: Exploring Pairwise Comparisons for LLM-Based Peer Review (NeurIPS 2025 · similarity 0.84)
- AAAR-1.0: Assessing AI’s Potential to Assist Research (ICML 2025 · similarity 0.84)
- Agent Reviewers: Domain-specific Multimodal Agents with Shared Memory for Paper Review (ICML 2025 · similarity 0.83)
- Retro*: Optimizing LLMs for Reasoning-Intensive Document Retrieval (ICLR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
