Research map / Vision-language and multimodal / Multimodal LLMs

TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TinyChemVL, a 4B chemical VLM (InternVL2.5-4B backbone) that uses adaptive visual token pruning/merging to cut visual tokens to 1/16 of ChemVLM's, and introduces the ChemRxn-V reaction recognition/prediction benchmark.

Problem

Standard VLMs applied to chemistry waste computation on sparse molecular images with blank backgrounds and are evaluated mostly on molecule-level tasks, leaving reaction-level visual chemistry (recognition and product prediction from images) underexplored.

Approach

Adapts the Adaptive Token Sampler scoring with Top-K pruning in low-variance layers and Bipartite Soft Matching merging in high-variance layers, with proportional attention, inside an InternVL2.5-4B backbone, and trains on large chemistry image corpora including reaction-level tasks.

Key results

• TinyChemVL reaches 91.2% Avg Sim. and 77.4% Tani@1.0 on ChemOCR, above ChemVLM-8B (81.7 / 57.7) with about 1/8 to 1/16 of the visual tokens.
• On ChemRxn-V reaction prediction, TinyChemVL scores 78.9 Avg Sim. and 52.4 Tani@1.0, while the best baseline GPT-4o reaches 30.4 and 1.4.
• Inference throughput is 11.84 samples/s versus 7.41 for ChemVLM-8B, with average input tokens of 108 versus 896.

Surprising findings

• Reducing visual tokens from 16 to 4 per image only moderately degrades results (ChemOCR -1.2, ChemRxn-V recognition -3.2, prediction -2.3), though the authors say 4 tokens are insufficient for reactions.

Limitations

• Benchmark ChemRxn-V relies on rendered images from ORDerly rather than real-world reaction images.
• Reported comparisons to ChemVLM and ChemMLLM rely on numbers from prior work or manual evaluation, with partially missing entries (e.g., ChemMLLM ChemOCR '-').

Experimental setup

Datasets: ChEBI-20-MM MolGrapher MolScribe ORDerly EDU-CHEMC ChemMLLM training corpora ChemOCR img2smiles img2property Mol-instructions

Benchmarks: ChemOCR img2smiles img2property img2img property2img ChemRxn-V

Metrics: Avg Sim. Tanimoto@1.0 MSE Increased LogP Exact Match RDKit fingerprint similarity throughput (samples/s) average tokens training hours

Baselines: GPT-4V GPT-4o Qwen2.5-VL-7B LLaVA-v1.5-7B Phi-3.5-vision InternVL2.5-4B ChemVLM-8B ChemVLM-26B ChemDFM-X ChemMLLM

Builds on: InternVL2.5 Adaptive Token Sampler ToMe ChemVLM ChemMLLM

Closest papers by meaning

• ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area (AAAI 2025 · similarity 0.91)
• Omni-Mol: Multitask Molecular Model for Any-to-any Modalities (NeurIPS 2025 · similarity 0.88)
• MolVision: Molecular Property Prediction with Vision Language Models (NeurIPS 2025 · similarity 0.88)
• Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations (NeurIPS 2025 · similarity 0.88)
• ReactGPT: Understanding of Chemical Reactions via In-Context Tuning (AAAI 2025 · similarity 0.87)
• ChemPile: A 250 GB Diverse and Curated Dataset for Chemical Foundation Models (NeurIPS 2025 · similarity 0.86)