AI Paper Atlas

Research map / Large language models / Agents, code and math

CME–SpectrumBench: Can LLMs Analyze Condensed Matter Spectral Data?

Jin Gene Wong, Anjney Midha, Joseph Tennyson, Wei-Lin Chiang, Ion Stoica, Zhi-Xun Shen

NeurIPS 2026 · Large language models › Agents, code and math · approach: LLMs for scientific discovery · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces CME–SpectrumBench, an expert-designed benchmark of 190 condensed matter spectral data analysis questions for evaluating LLMs on experimental data analysis.

Problem

Existing LLM benchmarks emphasize theoretical, symbolic reasoning in math and physics, while experimental data analysis in frontier research (here, condensed matter spectroscopy) is largely unevaluated.

Approach

An expert-designed benchmark of 190 questions focused on analyzing condensed matter spectral data, targeting data analysis rather than symbolic theoretical reasoning.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
give the LLM a Python interpreter with or without ReAct prompting (ablation)best frontier model · CME-SpectrumBench, all spectraLorentzian score: 0.34 -> 0.46Improved
give the LLM a Python interpreter with or without ReAct prompting (ablation)best frontier model · CME-SpectrumBench, minimal noise/convolutionLorentzian score: 0.48 -> 0.68Improved
give the LLM a Python interpreter with or without ReAct prompting (ablation)frontier models · CME-SpectrumBench, maximum noise/convolutionLorentzian scoreNo clear effect
give the LLM a Python interpreter with or without ReAct prompting (ablation)Qwen3 · CME-SpectrumBenchLorentzian scoreNo clear effect
give the LLM a Python interpreter with or without ReAct prompting (ablation)Claude Opus and Sonnet 4.5 · CME-SpectrumBenchLorentzian scoreImproved
increase dataset resolution (token count) (ablation)Gemini 3.1 Pro, direct query · CME-SpectrumBench noiseless questionsnormalized scoreHurt
increase noise level (ablation)direct-query models · CME-SpectrumBenchLorentzian scoreHurt
add a small amount of noise at low noise levels (ablation)Gemini 2.5 Pro Preview, question B2 · CME-SpectrumBenchLorentzian scoreMixed
advanced prompting on Tier II array-extraction questions (ablation)frontier models · CME-SpectrumBench Tier IILorentzian scoreImproved

Experimental setup

Benchmarks: CME–SpectrumBench

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy