Research map / Image generation / Autoregressive and latent generation

All-atom Diffusion Transformers: Unified generative modelling of molecules and materials

ICML 2025 · Image generation › Autoregressive and latent generation · approach: Equivariant diffusion for 3D molecules · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ADiT, a unified latent diffusion framework (Transformer VAE plus DiT denoiser with flow matching) that jointly generates molecules and crystals from a shared all-atom latent space.

Problem

Generative models for atomic systems are highly specific to molecules or crystals, using equivariant multi-modal diffusion on categorical-continuous product manifolds; no unified model covers both periodic materials and non-periodic molecules.

Approach

Encode all-atom representations (atom types, 3D coordinates, fractional coordinates, lattice parameters) of molecules and crystals into a shared latent space with a standard Transformer VAE, then run a non-equivariant DiT flow-matching model in that latent space with classifier-free guidance on the periodic/non-periodic label.

Key results

• Jointly trained ADiT achieves 5.3% S.U.N. rate on MP20 crystals versus 3.3% (DiffCSP) and 2.8% (FlowMM, replicated), and 91.92% overall validity.
• On QM9, jointly trained ADiT reaches 97.43% validity and 96.92% uniqueness, matching or exceeding equivariant diffusion baselines.
• ADiT generates 10,000 samples in under 20 minutes on a single V100 GPU, reported as an order of magnitude faster than baselines taking up to 2.5 hours.

Measured effects reported in the paper

Surprising findings

• Larger DiT denoisers improve stability and validity but reduce the S.U.N. rate on MP20, attributed to memorization of the small 27K-crystal training set.
• Standard non-equivariant Transformer autoencoders outperformed Equiformer-V2 equivariant autoencoders in both reconstruction and generative validity.

Limitations

• Relies on small datasets (MP20 27K crystals, QM9), limiting generalization and possibly causing memorization at larger model scale.
• MOF generation validity is lower (10-16%) and the joint model did not fully converge.
• Only unconditional generation is demonstrated; conditional/property-guided generation is left to future work.

Experimental setup

Datasets: QM9 MP20 GEOM-DRUGS QMOF

Benchmarks: MP20 QM9 GEOM-DRUGS QMOF

Metrics: validity uniqueness stability S.U.N. rate PoseBusters pass rate MOFChecker pass rate

Baselines: CDVAE DiffCSP FlowMM UniMat FlowLLM MatterGen-MP Equivariant Diffusion GeoLDM Symphony SemlaFlow EQGAT-diff

Builds on: Latent diffusion models Diffusion Transformers Classifier-free guidance GeoLDM Equivariant Diffusion

Closest papers by meaning

• Navigating the Design Space of Equivariant Diffusion-Based Generative Models for De Novo 3D Molecule Generation (ICLR 2024 · similarity 0.90)
• Scalable Diffusion for Materials Generation (ICLR 2024 · similarity 0.90)
• LLM Meets Diffusion: A Hybrid Framework for Crystal Material Generation (NeurIPS 2025 · similarity 0.90)
• Geometric-Facilitated Denoising Diffusion Model for 3D Molecule Generation (AAAI 2024 · similarity 0.90)
• Periodic Materials Generation using Text-Guided Joint Diffusion Model (ICLR 2025 · similarity 0.89)
• Chemistry-Inspired Diffusion with Non-Differentiable Guidance (ICLR 2025 · similarity 0.89)