Research map / Large language models / Agents, code and math
NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents
Tianshi Zheng, Kelvin Kiu Wai Tam, Newt Nguyen Kim Hue Nam, Baixuan Xu, Zhaowei Wang, Cheng Jiayang, Hong Ting Tsang, Weiqi Wang, Jiaxin Bai, Tianqing Fang, Yangqiu Song, Ginny Wong and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| give the weaker agent a code interpreter | GPT-4.1-mini · NewtonBench avg | symbolic accuracy (%): 5.1 -> 14.7 | Improved |
| give the weaker agent a code interpreter | GPT-4.1 · NewtonBench avg | symbolic accuracy (%): 5.8 -> 18.9 | Improved |
| give the weaker agent a code interpreter | DeepSeek-V3 · NewtonBench avg | symbolic accuracy (%): 8.9 -> 13.3 | Improved |
| give the stronger agent a code interpreter | GPT-5 · NewtonBench avg | symbolic accuracy (%): 75.9 -> 73.3 | Hurt |
| give the stronger agent a code interpreter | Gemini-2.5-flash · NewtonBench avg | symbolic accuracy (%): 50.6 -> 40.9 | Hurt |
| give the stronger agent a code interpreter | GPT-5-mini · NewtonBench avg | symbolic accuracy (%): 53.1 -> 48.1 | Hurt |
| give the stronger agent a code interpreter | Gemini-2.5-pro · NewtonBench avg | symbolic accuracy (%): 65.4 -> 63.7 | Hurt |
| give the agent a code interpreter | QwQ-32B · NewtonBench avg | symbolic accuracy (%): 28.5 -> 31.4 | Improved |
| use a reasoning model instead of non-reasoning (GPT-4.1-mini to GPT-5) | GPT family · NewtonBench avg | symbolic accuracy (%): 5.1 -> 75.9 | Improved |
| increase system complexity from vanilla equation to complex system, hard level | GPT-5 · NewtonBench | symbolic accuracy (%): 87.5 -> 40.3 | Hurt |
| add Gaussian observation noise of 0.0001 | GPT-5-mini · NewtonBench | symbolic accuracy: 12-16% reduction in accuracy | Hurt |
| increase noise level from 0.0001 to 0.1 | GPT-5-mini · NewtonBench | RMSLE: RMSLE remained relatively stable | No clear effect |
Closest papers by meaning
- PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors (NeurIPS 2025 · similarity 0.91)
- LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models (ICML 2025 · similarity 0.90)
- LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery (ICML 2024 · similarity 0.89)
- DiscoveryBench: Towards Data-Driven Discovery with Large Language Models (ICLR 2025 · similarity 0.88)
- SciAgentGym: Benchmarking Multi-Step Scientific Tool-Use in LLM Agents (ICML 2026 · similarity 0.87)
- MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
