Research map / Large language models / Agents, code and math
Large Language-Geometry Model: When LLM meets Equivariance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EquiLLM, which pairs a frozen GPT-2 LLM (fed geometry-aware text prompts and invariant features) with an equivariant encoder and adapter to model 3D molecular dynamics, human motion, and antibody CDR-H3 design while preserving E(3)-equivariance.
Problem
Predicting 3D structures and dynamics of physical systems requires E(3)-equivariance, but geometric GNNs cannot easily exploit broad external knowledge, while LLMs lack guaranteed spatial equivariance and cannot directly process continuous 3D coordinates.
Approach
Route only invariant features and prompts through a frozen LLM, keep 3D directional information in the equivariant encoder and an EGNN-style adapter, and use invariant statistics (distance-based) in prompts instead of absolute coordinates.
Key results
- On MD17, EquiLLM achieves MSE 2.391e-3 on aspirin vs ESTAG 3.263e-3 (-26.72%) and is best on all eight molecules.
- On Human Motion Walk R=20, EquiLLM reaches MSE 2.213e-2 vs ESTAG 3.464e-2, a 36.11% reduction.
- On RAbD antibody design, EquiLLM gets 38.97% AAR and 1.73 RMSD, beating GeoAB (36.43%, 1.79) on these metrics, though GeoAB has higher TM-score.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add EquiLLM (GPT-2 and geometry-aware prompting) to ESTAG | ESTAG on MD17 · MD17, all eight molecules | MSE: Reduction w.r.t ESTAG: -26.72%, -17.84%, -5.41%, -18.32%, -28.63%, -31.01%, -27.90%, -42.76% | Helped |
| add EquiLLM (GPT-2 and geometry-aware prompting) to ESTAG | ESTAG on Human Motion · Walk and Basketball, R=10, R=15, R=20 | MSE: -23.97%, -30.74%, -36.11%, -10.17%, -9.70%, -5.63% | Helped |
| add EquiLLM (GPT-2) to MEAN | MEAN on RAbD · RAbD | AAR: 36.77% -> 38.97 % | Helped |
| add EquiLLM (GPT-2) to MEAN | MEAN on RAbD · RAbD | TM-score: 0.9812 -> 0.9830 | Hurt |
| process raw features through LLM before EE and omit Equivariant Adapter and prompt (ablation) | EE (ESTAG) on MD17 · MD17, eight molecules | MSE | Mixed |
| add prompts to the LLM-before-EE variant without an adapter (ablation) | LLM (w/o Prompt) + EE on MD17 · MD17, eight molecules | MSE | Hurt |
| remove prompt (ablation) | EquiLLM on MD17 · MD17, eight molecules | MSE | Mixed |
| remove object feature description from prompt (ablation) | EquiLLM on MD17 · MD17, eight molecules | MSE | Mixed |
| remove statistics from prompt (ablation) | EquiLLM on MD17 · MD17, eight molecules | MSE | Mixed |
Surprising findings
- Directly prompting GPT-4o-mini, Gemini-1.5-flash, and DeepSeek-V3 to predict coordinates performs worse than most geometric GNN baselines.
- Adding the prompt on top of an LLM+encoder setup without the adapter significantly degraded performance in the ablation.
Limitations
- Uses GPT-2 as the LLM, which the authors note is a concept-validation choice; stronger LLMs not explored.
- Ablation shows a naive LLM+encoder without prompt design hurts performance, so gains depend on prompt design.
- GeoAB retains higher TM-score on antibody design, so EquiLLM is not best on all metrics.
Experimental setup
Datasets: MD17 Human Motion Capture (Walk, Basketball) SAbDab RAbD
Benchmarks: MD17 Human Motion Capture RAbD
Metrics: MSE AAR TM-score RMSD
Baselines: ST_GNN ST_TFN STGCN ST_SE(3)-Tr. ST_EGNN ESTAG GPT-4o-mini Gemini-1.5-flash-latest DeepSeek-V3 RosettaAD LSTM RefineGNN
Builds on: ESTAG MEAN EGNN LLaVA
Closest papers by meaning
- Equivariant Graph Neural Operator for Modeling 3D Dynamics (ICML 2024 · similarity 0.88)
- Scaling-Aware Adapter for Structure-Grounded LLM Reasoning (ICML 2026 · similarity 0.87)
- Geometric Hyena Networks for Large-scale Equivariant Learning (ICML 2025 · similarity 0.86)
- E(n) Equivariant Topological Neural Networks (ICLR 2025 · similarity 0.85)
- Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning (ICML 2024 · similarity 0.85)
- Neural P$^3$M: A Long-Range Interaction Modeling Enhancer for Geometric GNNs (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.