Research map / Large language models / Agents, code and math
CatalystBench: A Comprehensive Multi-Task Benchmark for Advancing Language Models in Catalysis Science
Xueqing Chen, Jian Xu, Ludi Wang, Yang Gao, Huihan Zhu, Yuanchun Zhou, Yi Du, Cheng-Lin Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add task-specific output heads to multi-task fine-tuning (MFT vs MT) (ablation) | ChemLLM · CatalystBench-test | avg improvement over single-task: 9.24% -> 12.44% | Improved |
| add multi-task training to task-specific output heads (MFT vs heads only) (ablation) | ChemLLM · CatalystBench-test | avg improvement over single-task: 5.13% -> 12.44% | Improved |
| fine-tune ChemLLM on catalytic data (CatalystLLM vs ChemLLM) | ChemLLM · CatalystBench FEP | R2: 0.45 -> 0.73 | Improved |
| fine-tune ChemLLM on catalytic data | ChemLLM · CatalystBench FP | MAE: 2.75 -> 1.34 | Improved |
| use CatalystLLM instead of gpt-4.1 | gpt-4.1 · CatalystBench AP | R2: 0.61 -> 0.81 | Improved |
| use CatalystLLM instead of gpt-4.1 | gpt-4.1 · CatalystBench SE | F1: 0.95 -> 0.94 | Hurt |
| use CatalystLLM instead of gpt-4.1 | gpt-4.1 · Experts score @100 | expert score: 0.56 -> 0.75 | Improved |
| use CatalystLLM instead of gpt-4.1 | gpt-4.1 · LLM score (gpt-4o judge) | LLM score: 0.85 -> 0.82 | Hurt |
| add regression, classification or extraction task data to training (ablation) | MFT · per-task performance | task performance | Improved |
| add RMC data to multi-task training (ablation) | MFT · FEP and RSC tasks | task performance | Improved |
| use structured complete input instead of unstructured core information (ablation) | CatalystLLM · CatalystBench | task performance | Improved |
Closest papers by meaning
- Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations (NeurIPS 2025 · similarity 0.88)
- Fine-Tuned Language Models Generate Stable Inorganic Materials as Text (ICLR 2024 · similarity 0.87)
- ReactGPT: Understanding of Chemical Reactions via In-Context Tuning (AAAI 2025 · similarity 0.87)
- MADE: Benchmark Environments for Closed-Loop Materials Discovery (ICML 2026 · similarity 0.86)
- Training a Scientific Reasoning Model for Chemistry (NeurIPS 2025 · similarity 0.86)
- Structured Chemistry Reasoning with Large Language Models (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
