Research map / Graph learning / Graph self-supervised learning
The quest for the GRAph Level autoEncoder (GRALE)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GRALE, a graph-level autoencoder that maps variable-size graphs to a fixed embedding using an Evoformer-based encoder/decoder and an OT-inspired, matching-free loss with a learnable Sinkhorn matcher.
Problem
Graph-level representation learning requires an autoencoder that encodes whole graphs of varying size into a single Euclidean embedding and decodes them back, without expensive graph-matching solvers in the loss and without ground-truth graph size at decoding.
Approach
Trains a graph-level autoencoder with an Optimal Transport-inspired reconstruction loss where the node matching is predicted by a differentiable Sinkhorn-based matcher trained end-to-end, using an Evoformer encoder and a novel Evoformer decoder with cross-attention and padded graphs so output size is predicted.
Key results
- On PUBCHEM 16 reconstruction, GRALE reaches 93.0% GI accuracy and 0.11 edit distance versus 41.0% and 1.69 for PIGVAE.
- On PUBCHEM 32, GRALE achieves 66.80% GI accuracy versus 24.91% for PIGVAE, and it obtains the best MLP regression MAE on QM9 (0.015).
- Graph matching with GRALE's matcher gives edit-distance upper bounds better than Pygmtools solvers on COLORING and PUBCHEM while running in about 0.008 s on PUBCHEM pairs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace OT loss with PIGVAE loss + regularization (ablation) | GRALE · PUBCHEM 16 | Edit Dist.: 0.11 -> 0.13 ± 0.16 | Hurt |
| replace second-order with first-order featurizer (ablation) | GRALE · PUBCHEM 16 | Edit Dist.: 0.11 -> 0.24±0.17 | Hurt |
| replace Evoformer encoder with GNN (ablation) | GRALE · PUBCHEM 16 | Edit Dist.: 0.11 -> 1.32 ± 0.16 | Hurt |
| replace Evoformer decoder with Transformer Decoder (ablation) | GRALE · PUBCHEM 16 | Edit Dist.: 0.11 -> 0.71±0.09 | Hurt |
| replace Sinkhorn matcher with Softsort (ablation) | GRALE · PUBCHEM 16 | Edit Dist.: 0.11 -> 1.47 ± 0.36 | Hurt |
| remove disambiguation noise (ablation) | GRALE · PUBCHEM 16 | Edit Dist.: 0.11 -> 1.16±0.03 | Hurt |
| finetune the pretrained GRALE model | GRALE · COLORING 15 | Edit Dist.: 0.67 -> 0.45 | Helped |
| finetune the pretrained GRALE model | GRALE · QM9 | Edit Dist.: 3.62 -> 2.13 | Helped |
| pretrain GRALE representations for downstream molecular prediction | GRALE · BBBP | ROC-AUC | Hurt |
| pretrain GRALE representations for downstream molecular prediction | GRALE · ESOL | MAE | Mixed |
Surprising findings
- The loss can be zero for non-isomorphic graphs under the PIGVAE relaxation, motivating the OT-based loss.
- Reconstruction performance and downstream task performance are highly correlated across pretraining dataset sizes and embedding dimensions.
Limitations
- Cubic computational complexity restricts graphs to N=32 in experiments.
- Matcher operates out of distribution on random graph pairs, requiring fine-tuning.
- Failure on highly symmetric graphs where matching is ambiguous.
Experimental setup
Datasets: COLORING PubChem MoleculeNet QM9 GDB13 BN dataset
Benchmarks: COLORING PUBCHEM 16 PUBCHEM 32 ESOL LIPO FreeSolv QM9 QM40 BBBP BACE COLORING 10 COLORING 15
Metrics: graph edit distance GI accuracy MAE ROC-AUC RMSE Pearson r compute time
Baselines: GraphVAE PIGVAE GVAE SimGRACE InfoGraph GraphMAE VGAE Any2Graph Relationformer FGWBARY IPFP RRWM
Builds on: Any2Graph PIGVAE AlphaFold Evoformer Sinkhorn
Closest papers by meaning
- HC-GAE: The Hierarchical Cluster-based Graph Auto-Encoder for Graph Representation Learning (NeurIPS 2024 · similarity 0.89)
- GraphCroc: Cross-Correlation Autoencoder for Graph Structural Reconstruction (NeurIPS 2024 · similarity 0.88)
- UniGTE: Unified Graph–Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains (NeurIPS 2025 · similarity 0.87)
- Self-supervised Masked Graph Autoencoder via Structure-aware Curriculum (ICML 2025 · similarity 0.87)
- Teacher-guided Edge Discriminator for Personalized Graph Masked Autoencoder (AAAI 2025 · similarity 0.87)
- Rethinking Graph Masked Autoencoders through Alignment and Uniformity (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.