Research map / Label-efficient and robust learning / Self-supervised and contrastive
ZeoBench: A Benchmark for Self-Supervised Learning on 3D Zeolite Representations
Aaron Sun, Yachan Liu, Ping Yang, Peng Bai, Subhransu Maji
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ZeoBench, a benchmark for evaluating self-supervised learning on 3D zeolite crystal structure representations, motivated by scarce property labels for zeolite structure-molecule pairs.
Problem
Identifying zeolite materials optimized for a target application (gas storage, separations, catalysis) is hard because property labels are expensive and available only for limited structure-molecule pairs, despite hundreds of thousands of known zeolite structures.
Approach
A benchmark for self-supervised learning on 3D zeolite representations, aimed at leveraging large unlabeled zeolite structure collections to reduce reliance on scarce property labels. Details beyond the abstract are not available.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| in-domain SSL pretraining before fine-tuning instead of training from scratch | 3D ResNet-18 · ZeoBench C18, 10k labels | log(kH) MSE: 6.2 -> 4.9 (SimCLR) | Improved |
| pretraining benefit grows as labels decrease | 3D ResNet-18 · ZeoBench fine-tuning, varying subset sizes | log(kH) MSE | Improved |
| use contrastive objectives (MoCo, SimCLR) instead of BYOL or DINO for frozen features | 3D ResNet-18 linear probe · ZeoBench linear probe, 10k labels | log(kH) MSE: 27.4 (MoCo) vs 34.3 (BYOL) vs 44.1 (DINO) | Improved |
| fine-tune after pretraining: BYOL versus contrastive methods | 3D ResNet-18 · ZeoBench fine-tuning, 10k labels | log(kH) MSE: BYOL 12.4 vs MoCo 12.6 | No clear effect |
| DINO pretraining before fine-tuning | 3D ResNet-18 · ZeoBench fine-tuning, 10k labels | log(kH) MSE: 13.5 -> 13.4 | No clear effect |
| add scale augmentation (sigma > 0) during SSL pretraining (ablation) | SSL pretraining · ZeoBench C18 10k linear probe | log(kH) MSE | Improved |
| vary scale augmentation strength sigma (ablation) | SSL pretraining · ZeoBench C18 10k linear probe | log(kH) MSE | No clear effect |
| use CNN (ResNet) instead of ViT or Swin trained from scratch (ablation) | 3D architectures · ZeoBench fine-tuning, 10k labels | log(kH) MSE: ViT 40.4, Swin 38.4 -> ResNet 13.5 | Improved |
| adapt ImageNet 2D ViT to 3D via depth tokenization (ablation) | linear probe baselines · ZeoBench linear probe, 10k labels | log(kH) MSE: 56.9 (handcrafted) / 60.8 (3DINO) -> 48.7 | Improved |
| use test-time augmentation averaging for ImageNet adaptation (ablation) | ImageNet depth tokenization · ZeoBench linear probe | log(kH) MSE | Improved |
| handcrafted descriptors in low-label regimes | handcrafted MLP baseline · ZeoBench linear probe, low labels | log(kH) MSE | Hurt |
| transfer ZeoBench SimCLR pretraining to hMOF adsorption | ResNet-18 · hMOF, 5,000 training MOFs | average MAE (mol/kg): 0.375 -> 0.242 | Improved |
Experimental setup
Benchmarks: ZeoBench
Closest papers by meaning
- A Denoising Pre-training Framework for Accelerating Novel Material Discovery (AAAI 2025 · similarity 0.87)
- A Diffusion-Based Pre-training Framework for Crystal Property Prediction (AAAI 2024 · similarity 0.85)
- Learning Over Molecular Conformer Ensembles: Datasets and Benchmarks (ICLR 2024 · similarity 0.84)
- DR-Label: Label Deconstruction and Reconstruction of GNN Models for Catalysis Systems (AAAI 2024 · similarity 0.84)
- Offline Materials Optimization with CliqueFlowmer (NeurIPS 2026 · similarity 0.84)
- MOFFlow: Flow Matching for Structure Prediction of Metal-Organic Frameworks (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
