Research map / Architectures and efficiency / Attention and transformers
Modeling Structure with Undirected Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add iterative energy updates (k=3) to joint classification and reconstruction training | Convolutional UNN · MNIST | accuracy: 98.80 -> 98.83 | Helped |
| Add reconstruction training to classification (ablation) | Convolutional UNN · MNIST | accuracy: 98.82 -> 98.75; 98.74 -> 98.78 | Mixed |
| Use tied weights across unrolled layers instead of untied weights (ablation) | Convolutional UNN · MNIST | accuracy: 98.76 -> 98.74; 98.45 -> 98.83; 98.32 -> 98.78; 97.39 -> 98.69 | Helped |
| Increase structured inference iterations beyond k=1 (ablation) | Structured UNN parser · Universal Dependencies languages | unlabeled attachment score: 89.09 -> 88.98; 85.62 -> 84.94; 93.79 -> 93.83; 91.96 -> 91.99; 83.41 -> 83.37; 85.11 -> 85.77; 94.76 -> 94.59; 96.99 -> | Mixed |
| Initialize the output randomly rather than with zeros (ablation) | Convolutional UNN · MNIST | accuracy: 98.83 -> 98.90 | No clear effect |
| Initialize the output uniformly rather than with zeros (ablation) | Convolutional UNN · MNIST | accuracy: 98.83 -> 98.83 | No clear effect |
| Use a forward-backward update order instead of random variable-update order (ablation) | Undirected attention · Sequence completion | test accuracy | Helped |
| Train with a random number of iterations instead of a fixed iteration count (ablation) | Undirected attention · Sequence completion | test accuracy | No clear effect |
| Use multiple energy updates to generate class prototypes instead of a single iteration (ablation) | Convolutional UNN · MNIST digit prototype generation | prototype clarity | Helped |
Closest papers by meaning
- Non-parametric Structured Output Networks (NeurIPS 2017 · similarity 0.87)
- Structured Convolutions for Efficient Neural Network Design (NeurIPS 2020 · similarity 0.86)
- Contextual Structure Knowledge Transfer for Graph Neural Networks (AAAI 2025 · similarity 0.85)
- Graph Structure Inference with BAM: Neural Dependency Processing via Bilinear Attention (NeurIPS 2024 · similarity 0.85)
- Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.84)
- How Transformers Learn Structured Data: Insights From Hierarchical Filtering (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.