AI Paper Atlas

Research map / Large language models / Retrieval and knowledge

InnoEval: On Research Idea Evaluation as a Knowledge-Grounded, Multi-Perspective Reasoning Problem

Shuofei Qiao, Yunxiang Wei, Xuehai Wang, Bin Wu, Boyang XUE, Ningyu Zhang, Hossein A. Rahmani, Wang Yanshan, Qiang Zhang, Keyan Ding, Jeff Z. Pan, Huajun Chen and 1 more

ICML 2026 · Large language models › Retrieval and knowledge · approach: Domain-specific QA datasets · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use full InnoEval pipeline instead of strongest baseline ScholarEvalDeepSeek-V3.2 · point-wise 3-classmacro F1 (%): 58.38 -> 74.56Improved
use InnoEval instead of ScholarEvalDeepSeek-V3.2 · group-wise best selectionaccuracy (%): 49.42 -> 65.12Improved
use InnoEval instead of ScholarEvalDeepSeek-V3.2 · pair-wise hardaccuracy (%): 60.00 -> 63.00Improved
add grounding agent that filters retrieved reports (ablation)InnoEval · point/pair/group-wiseaccuracy and F1Improved
add personalized reviewer personas instead of plain LLM-as-judge (ablation)InnoEval · point-wise and group-wiseaccuracy and F1Improved
add web and code search beyond literature only (ablation)InnoEval · pair-wise and group-wiseaccuracyImproved
swap backbone DeepSeek-V3.2 for o4-mini (ablation)InnoEval · all tasksaccuracy and F1Mixed
add diverse academic personas vs vanilla test-time scaling (ablation)InnoEval · point-wise and group-wiseF1 / accuracyImproved
use a single persona instead of no persona (ablation)InnoEval · group-wise rankingaccuracyHurt
use InnoEval feedback in ResearchAgent idea iterationResearchAgent · idea generation qualityaverage scoreImproved
use ScholarEval feedback in ResearchAgent idea iterationResearchAgent · idea generation qualityaverage scoreHurt
use ScholarEval search engine instead of InnoEval'sevaluation · search qualitydiversityHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy