Research map / Multi-agent systems and games / LLM agents

From What to Why: A Multi-Agent System for Evidence-based Chemical Reaction Condition Reasoning

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ChemMAS, a multi-agent LLM system (Qwen3-8B backbone, SFT + GRPO with tool-use reward) that recommends reaction conditions with mechanistic, evidence-based rationales, reporting large Top-k similarity gains over specialized models and general LLMs.

Problem

Reaction condition recommendation mainly predicts what conditions to use without explaining why they are appropriate, limiting trust in high-stakes chemical workflows.

Approach

Reframes condition prediction as evidence-based reasoning: a General Chemist agent grounds mechanism, multi-channel recall builds a 5k candidate pool, and agents run tournament-style pairwise debates with multi-step tool-augmented reasoning, trained via SFT then GRPO with a bonus for combined search and memory tool use.

Key results

• On the private dataset, ChemMAS reaches 78.1% Top-1 catalyst similarity vs 63.4% for the best zero-shot LLM (Gemini 2.5 Pro).
• On ChemCoTBench RCR, ChemMAS achieves 62.1% Top-1 catalyst similarity, +16.5 over Gemini 2.5 Pro.
• Interpretability LLM-Score of 92.8 vs 62.5–77.2 for DeepSeek-R1 and GPT-5 baselines; General Chemist accuracies 95.8% (MainFG), 90.2% (by-product), 92.5% (reaction type).

Surprising findings

• Zero-shot general LLMs given only SMILES reach roughly 5% Top-1 similarity, so the paper evaluates them in a candidate-ranking regime over a shared 5k pool.
• Ablating SFT hurts more than ablating RL, and removing the accuracy term (Acc) yields higher scores on several columns than the full model.

Limitations

• Evaluation relies on a private dataset that is not released, limiting reproducibility.
• General LLM baselines are evaluated in a candidate-ranking regime over a retrieval pool built by the authors' own recall module, which may favor the proposed system.
• Rationale quality is judged by an LLM-as-a-judge and a small human study, with no detailed reporting of the judge model or human protocol.

Experimental setup

Datasets: private organic reaction dataset (544,591 entries) ChemCoTBench RCR subset (90 instances)

Benchmarks: private organic reaction test set ChemCoTBench RCR subset

Metrics: Top-k Tanimoto similarity (k=1,5,10) BLEU-4 LLM-Score accuracy of intermediate outputs

Baselines: RCR Reagent Transformer MM RCR Qwen3-235B-A22B GPT-5 Claude 3.7 Sonnet DeepSeek-R1 Gemini 2.5 Pro

Builds on: GRPO Tool-Integrated Reasoning

Closest papers by meaning

• Can Agents Price a Reaction? Evaluating LLMs on Chemical Cost Reasoning (NeurIPS 2026 · similarity 0.90)
• Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations (NeurIPS 2025 · similarity 0.89)
• Training a Scientific Reasoning Model for Chemistry (NeurIPS 2025 · similarity 0.88)
• LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning (ICML 2026 · similarity 0.87)
• ReactGPT: Understanding of Chemical Reactions via In-Context Tuning (AAAI 2025 · similarity 0.87)
• MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses (ICLR 2025 · similarity 0.85)