Research map / Graph learning / GNN architectures
Deriving Neural Architectures from Sequence and Graph Kernels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace fixed λ = 0.8 with a learned decay parameter (ablation) | Kernel NN · PTB test | perplexity: 84.3 -> 76.8 | Helped |
| Use an input-adaptive gated λ instead of a fixed or learned scalar decay (ablation) | Kernel NN · PTB test | perplexity: 76.8 -> 73.6 | Helped |
| Add variational dropout (ablation) | Kernel NN (gated λ), 20m parameters · PTB test | perplexity: 69.2 -> 65.5 | Helped |
| Increase recurrence depth to 4 layers (ablation) | Kernel NN (gated λ) + variational dropout · PTB test | perplexity: 65.5 -> 63.8 | Helped |
| Use constant λ instead of a gate (ablation) | Kernel NN · SST fine-grained test | accuracy: 51.2 -> 53.2 | Hurt |
| Use constant λ instead of a gate (ablation) | Kernel NN · SST binary test | accuracy: 88.6 -> 89.9 | Hurt |
| Use input-dependent decay gates (ablation) | Weisfeiler Kernel NN · Harvard Clean Energy Project test | RMSE: 0.1058 -> 0.1043 | Helped |
| Include non-contiguous n-gram patterns with recurrence (λ > 0), rather than the λ = 0 word-level CNN case (ablation) | Kernel NN · PTB test | perplexity | Helped |
Closest papers by meaning
- Kernel-Based Approaches for Sequence Modeling: Connections to Neural Methods (NeurIPS 2019 · similarity 0.89)
- Recurrent Kernel Networks (NeurIPS 2019 · similarity 0.87)
- Convolutional Sequence to Sequence Learning (ICML 2017 · similarity 0.87)
- Separations in the Representational Capabilities of Transformers and Recurrent Architectures (NeurIPS 2024 · similarity 0.86)
- How Graph Neural Networks Learn: Lessons from Training Dynamics (ICML 2024 · similarity 0.85)
- A Tensor Decomposition Perspective on Second-order RNNs (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.