AI Paper Atlas

Research map / Large language models / Agents, code and math

HeurekaBench: A Benchmarking Framework for AI Co-scientist

Siba Smarak Panigrahi, Jovana Videnović, Maria Brbic

ICLR 2026 · Large language models › Agents, code and math · approach: Reasoning benchmarks and platforms · 12 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add end-critic (ablation)Biomni (GPT-OSS-120B) · sc-HeurekaBench OEQscorrectness: 2.08 -> 2.40Improved
add end-critic (ablation)Biomni (Qwen3-235B-thinking) · sc-HeurekaBench OEQscorrectness: 1.85 -> 1.73Hurt
add plan-critic (ablation)Biomni (GPT-OSS-120B) · sc-HeurekaBench OEQscorrectness: 2.08 -> 1.92Hurt
add plan-critic (ablation)Biomni (Qwen3-235B-thinking) · sc-HeurekaBench OEQscorrectness: 1.85 -> 1.56Hurt
add retriever module (ablation)Biomni (GPT-OSS-120B) · sc-HeurekaBench-TUcorrectness: 1.56 -> 2.15Improved
add retriever module (ablation)Biomni (Qwen3-235B-thinking) · sc-HeurekaBench-TUcorrectness: 1.80 -> 1.92Improved
use Claude-4-Sonnet instead of GPT-OSS-120B as plannerBiomni · sc-HeurekaBench OEQscorrectness: 2.08 -> 2.58Improved
use thinking variant of Qwen3-235B as planner (ablation)Biomni · sc-HeurekaBench OEQscorrectness: 1.57 -> 1.85Improved
scale Qwen3 planner from 32B to 235B (ablation)Biomni · sc-HeurekaBench OEQscorrectness: 1.47 -> 1.57Improved
use GPT-OSS-120B instead of Qwen3-235B as plannerBiomni · sc-HeurekaBench OEQscorrectness: 1.57 -> 2.08Improved
use Biomni instead of CellVoyagerClaude-4-Sonnet agent · sc-HeurekaBench-Lite MCQsaccuracy: 27.78 -> 50.00Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy