Research map / Continual learning / Continual fine-tuning of LLMs
GeoAda: Efficiently Finetune Geometric Diffusion Models with Equivariant Adapters
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GeoAda, an SE(3)-equivariant adapter (coupling/decoupling operators, trainable layer copies, equivariant zero-convolution) that parameter-efficiently fine-tunes frozen geometric diffusion models for frame, global, and subgraph controls across particle, molecular dynamics, human motion, and molecule generation tasks.
Problem
Fine-tuning pretrained SE(3)-equivariant geometric diffusion models to new downstream tasks with varying geometric controls is inefficient and prone to overfitting and catastrophic forgetting of the pretraining task.
Approach
Adapt a frozen geometric diffusion denoiser with ControlNet-style trainable copies of selected layers, wrapped by control-specific coupling/decoupling operators and an SE(3)-equivariant zero-initialized convolution, so the conditional score is equivariant and zero at initialization.
Key results
- On Charged Particles, GeoAda achieves ADE 1.119 vs Full FT 1.093 (unconditional FT) and retains pretrain performance (Marg 0.079 vs pretrain 0.079) while GPF and others degrade.
- On MD17 FT tasks, GeoAda outperforms baselines by an average of 18.94% in ADE and 18.22% in FDE, while retaining pretrain-task accuracy.
- On CMU Mocap long-term prediction, GeoAda reports the best downstream ADE on running (60.88 vs 61.92 for GPF) and walking (34.52 vs 41.31 for GPF-plus).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GeoAda instead of Full FT for downstream frame control | Charged Particles trajectory generation · Charged Particles conditional FT | ADE: 1.106 -> 1.105 | Helped |
| Use GeoAda instead of prompt-based and partial fine-tuning baselines | GeoAda · Charged Particles trajectory generation | ADE: achieving 35.69% improvement on ADE and 21.29% on FDE | Helped |
| Use GeoAda instead of Full FT | MD17 molecular dynamics prediction · MD17 Aspirin | ADE: 0.929 -> 0.891 | Helped |
| Use GeoAda instead of Full FT | MD17 molecular dynamics prediction · MD17 Benzene | ADE: 0.217 -> 0.191 | Helped |
| Use GeoAda instead of Full FT | CMU Mocap motion prediction · CMU Mocap walking 80 ms | prediction error: 10.01 -> 8.92 | Helped |
| Replace equivariant zero convolution with Gaussian-initialized standard convolutions (ablation) | GeoAda adapter · Fine-tuning tasks | performance | Hurt |
| Replace each trainable copy with a single convolution layer (ablation) | GeoAda adapter · Fine-tuning tasks | performance | Hurt |
| Increase the number of trainable copy layers (ablation) | GeoAda adapter · Fine-tuning tasks | performance | Helped |
| Select the first layer for every K consecutive layers rather than naive initial- or last-layer selections (ablation) | GeoAda adapter · Fine-tuning tasks | performance | Helped |
Surprising findings
- Full fine-tuning and prompt-based methods often produce invalid (nan) samples on the original pretraining task, indicating severe catastrophic forgetting.
- A sudden convergence phenomenon is observed during adapter training, with loss and validation MSE dropping abruptly between steps 4500 and 4700.
Limitations
- Full fine-tuning baseline and PEFT comparisons are limited to a few geometric domains and small datasets
- Adapter performance depends on choice of which pretrained layers are copied (ablation-driven heuristic)
Experimental setup
Datasets: Charged Particles MD17 CMU Mocap QM9 GEOM-Drugs CrossDocked2020
Benchmarks: Charged Particles MD17 CMU Mocap CrossDocked2020
Metrics: ADE FDE Marginal score Classification score Prediction score Vina Score Vina Min Vina Dock High Affinity QED
Baselines: Full FT PARTIAL-k GPF GPF-plus Prompt-Template MLP-k liGAN GraphBP AR Pocket2Mol TargetDiff
Builds on: ControlNet AdapterGNN GPF GeoDiff
Closest papers by meaning
- Geometric Trajectory Diffusion Models (NeurIPS 2024 · similarity 0.90)
- Equivariant Neural Diffusion for Molecule Generation (NeurIPS 2024 · similarity 0.88)
- Equivariant Blurring Diffusion for Hierarchical Molecular Conformer Generation (NeurIPS 2024 · similarity 0.88)
- Bridging Geometric States via Geometric Diffusion Bridge (NeurIPS 2024 · similarity 0.88)
- Scalable Non-Equivariant 3D Molecule Generation via Rotational Alignment (ICML 2025 · similarity 0.87)
- GeoSPRINT: Geometric Redundancy-Aware Step Pruning for Inference in Diffusion Trajectories (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.