Research map / Architectures and efficiency / RNNs and normalization
A Walsh Hadamard Derived Linear Vector Symbolic Architecture
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Hadamard-derived linear Binding (HLB), a Walsh-Hadamard-derived vector symbolic architecture with O(d) binding and unbinding, and evaluates it on classic VSA retrieval tasks and two deep learning tasks (CSPS and XML classification).
Problem
Existing vector symbolic architectures (VSAs) are either computationally expensive, numerically unstable (HRR/FFT), or underperform in differentiable deep learning systems.
Approach
Derives a VSA by replacing the Fourier transform in HRR circular convolution with the Walsh-Hadamard transform; the Hadamard projection cancels out, leaving element-wise multiplication for binding and element-wise division for unbinding, with a MiND initialization keeping values away from zero and a sqrt(rho) similarity correction for bundled items.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace HRR, VTB, or MAP binding with HLB | CSPS classification · MNIST, SVHN, CR10, and CR100 | Top@1 accuracy: 98.51 -> 98.73; 92.43 -> 94.53; 82.83 -> 83.81; 57.76 -> 58.82 | Helped |
| Replace HRR, VTB, or MAP binding with HLB | XML classification · All eight XML datasets | nDCG | Helped |
| Replace HRR, VTB, or MAP binding with HLB | XML classification · All eight XML datasets | PSnDCG | Helped |
| Add the input projection step (ablation) | HLB retrieval from bundled representations · Varying dimension and number of bound vector pairs | retrieval noise: 19.38 -> 7.18 | Helped |
| Use the HLB unbinding operation for sequential binding | Classical random and auto-binding tasks · Repeated sequential binding | similarity score for a present item | Helped |
Surprising findings
- VTB collapses to 19.59% Top@1 on SVHN in CSPS, far below other methods.
- MNIST clustering results for information hiding were degenerate, with ARI values showing leakage for several methods, which the authors attribute to CSPS security being heuristic.
Limitations
- Hadamard dimensions must be powers of two (d = 2^n).
- CSPS privacy is heuristic rather than guaranteed, as shown by MNIST clustering leakage.
- Noise terms in the unprojected form are not eliminated; the projected form's advantage relies on expectation arguments and empirical checks.
Experimental setup
Datasets: MNIST SVHN CIFAR-10 CIFAR-100 Mini-ImageNet Bibtex Delicious Mediamill EURLex-4K EURLex-4.3K Wiki10-31K Amazon-13K
Benchmarks: MNIST SVHN CIFAR-10 CIFAR-100 Mini-ImageNet Bibtex Delicious Mediamill EURLex-4K EURLex-4.3K Wiki10-31K Amazon-13K
Metrics: Top@1 accuracy Top@5 accuracy adjusted rand index nDCG PSnDCG area under accuracy curve cosine similarity
Baselines: HRR VTB MAP-C MAP-B
Builds on: holographic reduced representations multiply add permute vector-derived transformation binding connectionist symbolic pseudo secrets TPR
Closest papers by meaning
- How Do Transformers Learn Variable Binding in Symbolic Programs? (ICML 2025 · similarity 0.82)
- Hadamard Representation: Scaffolding Performance Across Model-free RL (NeurIPS 2026 · similarity 0.82)
- BlendRL: A Framework for Merging Symbolic and Neural Policy Learning (ICLR 2025 · similarity 0.80)
- OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems (NeurIPS 2026 · similarity 0.80)
- Visual symbolic mechanisms: Emergent symbol processing in Vision Language Models (ICLR 2026 · similarity 0.80)
- DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning (ICML 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.