Research map / Large language models / Language, safety and interpretability

ReactGPT: Understanding of Chemical Reactions via In-Context Tuning

AAAI 2025 · Large language models › Language, safety and interpretability · approach: Data and evaluation methodology · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ReactGPT, which adapts LLaMA3-8B-Instruct to chemical reaction captioning via fingerprint-based retrieval of context examples, domain-specific prompts, in-context tuning, and DPO-based structure preference alignment.

Problem

Aligning chemical reactions with natural language descriptions (reaction captioning and experimental procedure prediction) remains underexplored, since prior work focused on single molecules and metrics like BLEU poorly capture valid reaction descriptions.

Approach

Retrieve similar reactions using rxnfp reaction fingerprints as in-context examples, fine-tune an LLM with in-context tuning, then apply DPO with rule-based structure-discriminator preference pairs to improve caption structure.

Key results

• ReactGPT reaches 36.41 BLEU-4 and 89.53% structure success on reaction captioning, versus 32.23 BLEU-4 and 76.56% for naive-SFT LLaMA3-8B-Instruct.
• On experimental procedure prediction, ReactGPT gets 43.73 BLEU-4 and 100% validity, roughly matching ReactXT (44.00 BLEU-4).
• Ablation: removing retrieval drops BLEU-4 from 36.41 to 33.91; removing in-context tuning drops it to 32.23.

Measured effects reported in the paper

Surprising findings

• GPT-4o-mini with one context example performs worst on reaction captioning (BLEU-4 21.36), below the nearest-neighbor baseline on structure success.
• Both too few (k=1) and too many (k=3) context examples underperform k=2.

Limitations

• Text metrics like BLEU and ROUGE are acknowledged as insufficient for many valid phrasings of a reaction.
• Structure discriminator is rule-based (keyword checks), which may be coarse.
• Experiments are limited to a single dataset (OpenExp).

Experimental setup

Datasets: OpenExp USPTO-Applications ORD

Benchmarks: OpenExp reaction captioning test split OpenExp experimental procedure prediction test split

Metrics: BLEU-2 BLEU-4 ROUGE-1 ROUGE-2 ROUGE-L METEOR structure success rate 100%LEV 90%LEV 75%LEV

Baselines: Nearest Neighbor MolT5-Base MolT5-Large GPT-4o-mini Galactica ChemDFM LLaMA3 ReactXT MolCA

Builds on: in-context tuning direct preference optimization MolReGPT ICMA ReactXT

Closest papers by meaning

• Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations (NeurIPS 2025 · similarity 0.88)
• LLM-Augmented Chemical Synthesis and Design Decision Programs (ICML 2025 · similarity 0.87)
• From What to Why: A Multi-Agent System for Evidence-based Chemical Reaction Condition Reasoning (ICLR 2026 · similarity 0.87)
• TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks (AAAI 2026 · similarity 0.87)
• ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area (AAAI 2025 · similarity 0.86)
• Omni-Mol: Multitask Molecular Model for Any-to-any Modalities (NeurIPS 2025 · similarity 0.86)