Research map / Large language models / Language, safety and interpretability

LICO: Large Language Models for In-Context Molecular Optimization

ICLR 2025 · Large language models › Language, safety and interpretability · approach: LLM unlearning and benchmarks · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Extends a frozen-backbone-plus-LoRA Llama-2-7B with new x/y embedding layers and a prediction head, trained on intrinsic and GP-synthetic molecular functions for in-context surrogate modeling in PMO black-box molecular optimization.

Problem

Black-box molecular optimization needs a surrogate that predicts properties from few observations; prompting LLMs in natural language is impractical for scientific domains with scarce pretraining data and long verbose contexts.

Approach

Add separate embedding layers for molecular fingerprints and scores plus a prediction head to a pretrained LLM, and train it via autoregressive in-context prediction on a mix of molecular intrinsic property functions and Tanimoto-kernel GP synthetic functions, keeping natural-language task prompts and <x>/<y> tokens to guide in-context reasoning.

Key results

• On PMO-1K (1000 oracle calls), LICO achieves the highest sum of scores (11.71) across 23 tasks, ahead of MOLLEO (11.65) and Genetic GFN (11.56).
• On the original PMO (10000 calls), LICO reaches a sum of 14.708, behind Genetic GFN (15.678) and Augmented Memory (15.002).
• Ablation: training with intrinsic-only functions gives 3.010 vs 3.099 for the 0.1-synthetic mix on 5 tasks; synthetic-only gives 2.936.

Measured effects reported in the paper

Surprising findings

• Directly prompting GPT-4o for in-context molecular property prediction performs poorly compared with LICO as context grows.
• Llama-2 7B as backbone outperforms Nach0-base and T5-base, suggesting general LLMs beat chemistry-finetuned LLMs as in-context surrogates.

Limitations

• Assumes an accessible set of intrinsic functions, which may not exist outside molecular optimization.
• Does not update on downstream objective data, so it underperforms methods that fine-tune on real oracle data (Genetic GFN, Augmented Memory) on the PMO 10K setting.
• Only evaluated on molecular optimization; generality to other domains is untested.

Experimental setup

Datasets: ZINC 250K PMO benchmark objectives Guacamol objectives

Benchmarks: PMO PMO-1K

Metrics: AUC Average Top-10 AUC Average Top-1 AUC Average Top-100 negative log-likelihood mean squared error root mean squared calibration error

Baselines: GP BO Graph GA REINVENT Genetic GFN Augmented Memory MOLLEO TNP GPT-4o

Builds on: Bayesian optimization Graph GA MOLLEO FPT ExPT

Closest papers by meaning

• Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Tasks (ICML 2025 · similarity 0.89)
• Efficient Evolutionary Search Over Chemical Space with Large Language Models (ICLR 2025 · similarity 0.88)
• Omni-Mol: Multitask Molecular Model for Any-to-any Modalities (NeurIPS 2025 · similarity 0.88)
• Improving Large Molecular Language Model via Relation-aware Multimodal Collaboration (AAAI 2026 · similarity 0.87)
• Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains (ICML 2024 · similarity 0.87)
• MolTailor: Tailoring Chemical Molecular Representation to Specific Tasks via Text Prompts (AAAI 2024 · similarity 0.87)