Research map / Graph learning / Graph algorithms and clustering
Efficient Learning of Discrete-Continuous Computation Graphs
David Friede, Mathias Niepert
This paper tried a common change: Add dropout
add dropout residual connections (DR) · ListOps · task accuracy: 74.8 -> 96.1 Improved
What 795 accepted papers report for this change
- 549 improved
- 87 no clear effect
- 12 mixed
- 147 hurt
- Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference ICML 2026 · apply layer dropout to whole transformer blocks instead of attention and FFN sub-blocks separately · 20 TPP val loss, 271M · val loss: 2.377 -> 2.367Improved
- Quantifying the Uncertainty of Foundation Models with Singular Value Ensembles ICML 2026 · use MC Dropout · vision and SST-2 · accuracy: 86.3 -> 76.6 on Flowers102Hurt
- Motion Planning in Compressed Representation Spaces ICML 2026 · add nested dropout during training · WOMD guided generation · goal FDE: 4.145 -> 1.104Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add temperature matching (TM) (ablation) | dropout-residual Gumbel-softmax GNN · ListOps | task accuracy: 95.1 -> 96.1 | Improved |
| add temperature matching (TM) (ablation) | same model · ListOps | edge precision: 76.9 -> 82.3 | Improved |
| add dropout residual connections (DR) (ablation) | Gumbel-softmax GNN · ListOps | task accuracy: 74.8 -> 96.1 | Improved |
| add dropout residuals (ablation) | same model · ListOps | edge precision: 43.9 -> 82.3 | Improved |
| add both DR and TM (ablation) | same model · ListOps | task accuracy: 56.2 -> 96.1 | Improved |
| discrete-continuous model vs best baseline on task accuracy | Ours tau=1 · ListOps | task accuracy: 96.8 -> 96.1 | No clear effect |
| discrete-continuous model vs LSTM for extrapolation | Ours tau=1 · ListOps depth 8 | extrapolation accuracy: 83.7 -> 92.6 | Improved |
| discrete-continuous model vs LSTM for extrapolation | Ours tau=1 · ListOps depth 10 | extrapolation accuracy: 76.9 -> 86.9 | Improved |
| train on ground-truth graph instead of sequence | Ours · ListOps | task accuracy: 96.1 -> 99.8 | Improved |
| temperature tau=2 vs tau=1 | Ours · ListOps | edge precision: 82.3 -> 76.8 | Hurt |
| discrete-continuous model vs CoKE | Ours tau=4 · WordNet path query | H@10: 67.9 -> 64.3 | Hurt |
| discrete-continuous model vs CoKE | Ours tau=4 · Freebase path query | H@10: 77.7 -> 68.7 | Hurt |
Closest papers by meaning
- Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms (NeurIPS 2024 · similarity 0.86)
- On the Hardness of Training Deep Neural Networks Discretely (AAAI 2025 · similarity 0.84)
- Universal Approximation Theorem of Deep Q-Networks (ICML 2025 · similarity 0.84)
- Deep learning for continuous-time stochastic control with jumps (NeurIPS 2025 · similarity 0.84)
- Learning from Suboptimal Data in Continuous Control via Auto-Regressive Soft Q-Network (ICML 2025 · similarity 0.84)
- Discrete Codebook World Models for Continuous Control (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
