Research map / Large language models / Agents, code and math
CMT-Benchmark: A Benchmark for Condensed Matter Theory Built by Expert Researchers
Haining Pan, James V Roggeveen, Erez Berg, Juan Felipe Carrasquilla Alvarez, Debanjan Chowdhury, Surya Ganguli, Federico Ghimenti, Juraj Hasik, Henry S. Hunt, Hong-Chen Jiang, Mason Kamb, Ying-Jer Kao and 7 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use newer reasoning model GPT-5 instead of GPT-4o | LLM on CMT-Benchmark · CMT-Benchmark | Pass@1 (%): 2.0 -> 30.0 | Improved |
| Evaluate on VMC problems requiring critical judgment, any model | 17 LLMs · CMT-Benchmark (VMC problems) | Pass@1 (%): 0.0 for all models | No clear effect |
| Use GPT-5 on PEPS problems rather than DMRG problems | GPT-5 · CMT-Benchmark | Pass@1 (%): 0.0 (DMRG) -> 66.7 (PEPS) | Improved |
Closest papers by meaning
- Diffusion-warm sampling of the XY model enables fast thermalization at scale (NeurIPS 2026 · similarity 0.85)
- Predicting Ground State Properties: Constant Sample Complexity and Deep Learning Algorithms (NeurIPS 2024 · similarity 0.85)
- PhysElite: How Far Are LLMs from Solving Olympiad-Level Physics Problems? (NeurIPS 2026 · similarity 0.84)
- Scaling Physical Reasoning with the PHYSICS Dataset (NeurIPS 2025 · similarity 0.84)
- Learning the Electronic Hamiltonian of Large Atomic Structures (ICML 2025 · similarity 0.84)
- FlowMM: Generating Materials with Riemannian Flow Matching (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
