Research map / Large language models / Reasoning and chain of thought

Scaling-Aware Adapter for Structure-Grounded LLM Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Cuttlefish, a unified all-atom multimodal LLM whose connector (Scaling-Aware Patching plus Geometry Grounding Adapter) adaptively sizes structural query tokens and injects geometry-grounded tokens, evaluated on molecules, proteins, and nucleic acids.

Problem

Existing structure-aware LLMs are modality-specific and compress all-atom structures into fixed-length tokens or sequences, losing geometric detail for large entities and inducing structural hallucinations.

Approach

An instruction-conditioned gate selects anchor atoms via a cumulative probability mass threshold, soft distance-plus-bias assignment grows variable-size patches so token count scales with atom count, and cross-attention fusion blocks retrieve full geometric detail from EGNN node embeddings before injecting tokens into the LLM.

Key results

• On GEO-AT, Cuttlefish + Llama-3.1-8B-I reaches an average METEOR of 0.428 and BERTScore of 0.864, versus the best sequence-only reasoning baseline average METEOR of 0.227.
• On Mol-Instructions molecule tasks, Cuttlefish achieves the top result on most tasks, e.g., captioning METEOR 0.715 vs 0.707 for Mol-LLaMA-3.1, and design FTS 0.422 vs 0.392.
• In a matched-budget comparison, Cuttlefish at 2048 tokens reaches average BERTScore 0.862 vs 0.760 for the Q-Former at fixed 2048 tokens.

Surprising findings

• Q-Former variant with the same max token length performs competitively on molecules but degrades sharply on proteins and DNA/RNA (METEOR 0.218 and 0.174).
• Removing structure at inference still substantially outperforms sequence-only fine-tuning (average BERTScore 0.811 vs 0.745), suggesting lasting geometric priors from structure-aware training.

Limitations

• Gains on DNA/RNA are modest, and the paper notes nucleic acids are not the primary performance driver.
• Only sequence-derived nucleic acid structures are modeled; higher-order folding is excluded.
• Hallucination and reasoning claims rely on internal benchmarks (GEO-AT) built by the authors.

Experimental setup

Datasets: GEO-AT Mol-Instructions DNA-Chat RNA-QA Nucleotide Transformer benchmark RNAcentral

Benchmarks: GEO-AT Mol-Instructions DNA-Chat RNA-QA

Metrics: METEOR BERTScore ROUGE-L validity RDKit fingerprint similarity accuracy MCC hallucination rate answer rate F1

Baselines: Mol-Llama ProtChatGPT Prot2Chat 3D-MoLM ChatNT RNA-GPT Mol-Instructions ProtLLM ProteinGPT Q-Former

Builds on: Q-Former EGNN LLaVA MolCA

Closest papers by meaning

• Large Language-Geometry Model: When LLM meets Equivariance (ICML 2025 · similarity 0.87)
• Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance (ICML 2026 · similarity 0.87)
• Unveiling Fine-Grained Visual Traces: Evaluating MultiModal Interleaved Reasoning Chains in Multimodal STEM Tasks (NeurIPS 2026 · similarity 0.86)
• S$^2$-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance (CVPR 2026 · similarity 0.86)
• How RL Unlocks the Aha Moment in Geometric Interleaved Reasoning (ICML 2026 · similarity 0.86)
• DiGraphHal-Bench: Evaluating Multimodal Large Language Models on Complex Directed Graphs (CVPR 2026 · similarity 0.86)