Research map / Graph learning / Graph self-supervised learning
Neural Architecture Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Defines Neural Architecture Retrieval and proposes a motif-based macro-graph encoder with motif-level and graph-level contrastive/classification pre-training, evaluated on 12.5k real-world and 30k NAS architectures.
Problem
Retrieving existing neural architectures with designs similar to a query architecture, which is hard for standard graph pre-training because computational graphs vary widely in size and contain repeated motifs.
Approach
Encode each node's neighbourhood iteratively to find motifs, rebuild the computational graph as a macro graph of motif embeddings, and pre-train with motif-level contrastive learning (context graphs) followed by graph-level contrastive plus cross-entropy classification on model family labels.
Key results
- On real-world architectures, the proposed method reaches MRR Top-20 0.825 versus 0.737 (GCN) and 0.756 (GAT).
- On NAS architectures, the method reaches MAP Top-100 0.935 versus 0.858 (GCN) and 0.901 (GAT).
- Pre-training on NAS data then transferring to real-world architectures gives NDCG Top-100 0.706 versus 0.678 from scratch.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use motif-based graph splitting instead of direct graph embedding | Neural Architecture Retrieval · Real, Top-20 | MRR: 0.756 -> 0.825 | Helped |
| use motif-based graph splitting instead of direct graph embedding | Neural Architecture Retrieval · Real, Top-20 | MAP: 0.542 -> 0.593 | Helped |
| use motif-based graph splitting instead of direct graph embedding | Neural Architecture Retrieval · NAS, Top-20 | MAP: 0.941 -> 0.952 | Helped |
| use motif-based graph splitting instead of direct graph embedding | Neural Architecture Retrieval · NAS, Top-20 | MRR: 1.000 -> 1.000 | No clear effect |
| replace motif sampling with fixed node-number splitting (ablation) | Neural Architecture Retrieval · Real, Top-20 | MRR: 0.825 -> 0.807 | Hurt |
| replace motif sampling with fixed motif-number splitting (ablation) | Neural Architecture Retrieval · Real, Top-20 | MRR: 0.825 -> 0.817 | Hurt |
| replace motif sampling with random subgraph splitting (ablation) | Neural Architecture Retrieval · Real, Top-20 | MRR: 0.825 -> 0.801 | Hurt |
| remove graph-level contrastive learning (use CE only) (ablation) | CE+CL graph-level objective · Real, Top-20 | MRR: 0.825 -> 0.824 | No clear effect |
| remove cross-entropy classification (use CL only) (ablation) | CE+CL graph-level objective · Real, Top-20 | MRR: 0.825 -> 0.565 | Hurt |
| remove graph-level contrastive learning (use CE only) (ablation) | CE+CL graph-level objective · NAS, Top-20 | MAP: 0.952 -> 0.953 | No clear effect |
| remove cross-entropy classification (use CL only) (ablation) | CE+CL graph-level objective · NAS, Top-20 | MAP: 0.952 -> 0.750 | Hurt |
| initialize real-world training with NAS pre-training | Neural Architecture Retrieval · Real, Top-50 | MRR: 0.826 -> 0.838 | Helped |
Surprising findings
- Contrastive loss alone performs very poorly on real-world architectures (MRR about 0.57), and only helps when combined with cross-entropy.
Limitations
- Real-world labels for relevance are derived from coarse family/regex-extracted hints, so ground truth is heuristic.
- Baselines are only GCN and GAT; no comparison with stronger graph pre-training methods reported in the main text.
- Hardware and training compute are not reported.
Experimental setup
Datasets: Hugging Face and PyTorch Hub architectures (12,517 crawled) 30,000 DARTS-space NAS architectures
Benchmarks: real-world architecture retrieval set (12,517 models) NAS architecture retrieval set (30,000 DARTS architectures)
Metrics: MRR MAP NDCG
Baselines: GCN GAT
Builds on: GCN GAT DARTS
Closest papers by meaning
- AutoGFM: Automated Graph Foundation Model with Adaptive Architecture Customization (ICML 2025 · similarity 0.86)
- Building Optimal Neural Architectures using Interpretable Knowledge (CVPR 2024 · similarity 0.86)
- DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning (AAAI 2024 · similarity 0.86)
- Data-Augmented Curriculum Graph Neural Architecture Search under Distribution Shifts (AAAI 2024 · similarity 0.86)
- Disentangled Continual Graph Neural Architecture Search with Invariant Modular Supernet (ICML 2024 · similarity 0.85)
- Charting the Design Space of Neural Graph Representations for Subgraph Matching (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.