Research map / Architectures and efficiency / RNNs and normalization
An Explicitly Relational Neural Network Architecture
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use explicitly relational PrediNet representations instead of the baseline central modules | Relations Game, single-task held-out object sets · Relations Game | accuracy: over 90% accuracy on all tasks with both held-out object sets | Helped |
| Use PrediNet rather than the baseline central modules | Relations Game xoccurs task · xoccurs | accuracy: more than 10% | Helped |
| Use PrediNet rather than the baselines other than MHA | Relations Game colour / shape task · colour / shape | accuracy: 25% or more | Helped |
| Pre-train on the between task and freeze the central module | PrediNet · Relations Game, stripes; target AAB, ABA, ABB | accuracy: 90% | Helped |
| Pre-train on five column patterns, then transfer to five row patterns with the central module frozen | PrediNet · Relations Game, stripes; five row patterns | accuracy: 85% | Helped |
| Replace the baseline agent’s central fully connected layer with PrediNet | Atari agent · Atari 57-game suite | human-normalised score: comparable; very similar mean human-normalised scores for 53 out of 57 games | No clear effect |
| Replace the central fully connected layer with PrediNet | Atari agent · Atari 57-game suite | number of parameters: 5,098,515 -> 1,590,355 | Helped |
| Use three row-pattern curriculum tasks instead of three analogous column-pattern tasks | PrediNet and baseline architectures · Relations Game, target match rows | accuracy | Mixed |
| Increase the number of heads from 32 to 64 while keeping 16 relations (ablation) | PrediNet · Relations Game multi-task curriculum | accuracy | Helped |
| Reduce the number of heads to 16 and increase relations to 32 (ablation) | PrediNet · Relations Game multi-task curriculum | accuracy | Hurt |
| Prune a random subset of heads at test time (ablation) | PrediNet and MHA · colour / shape | accuracy | Mixed |
| Keep only heads that attend to both objects (ablation) | PrediNet · colour / shape | accuracy | No clear effect |
Closest papers by meaning
- A simple neural network module for relational reasoning (NeurIPS 2017 · similarity 0.89)
- When No Paths Lead to Rome: Benchmarking Systematic Neural Relational Reasoning (NeurIPS 2025 · similarity 0.88)
- Systematic Relational Reasoning With Epistemic Graph Neural Networks (ICLR 2025 · similarity 0.87)
- Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects (NeurIPS 2024 · similarity 0.87)
- When can transformers reason with abstract symbols? (ICLR 2024 · similarity 0.87)
- Disentangling and Integrating Relational and Sensory Information in Transformer Architectures (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.