Research map / Label-efficient and robust learning / OOD and anomaly detection
Rectified Lagrangian for Out-of-Distribution Detection in Modern Hopfield Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RegLag (rectified Lagrangian) for memory neurons in modern Hopfield networks, creating a trivial point attractor for OOD samples and training the interaction matrix via probabilistic interaction, evaluated for OOD detection on CIFAR-10/100 against energy-based baselines.
Problem
Modern Hopfield networks assume all inputs are in-distribution, so every test sample, including OOD samples, falls into an ID memory attractor; existing Hopfield-energy OOD detectors lack a probabilistic foundation.
Approach
A ReLU-rectified Lagrangian for memory neurons (log-sum-exp with inverse memory strength gamma, clipped at zero) creates a trivial point attractor at the origin for any interaction matrix, so samples falling into it are flagged OOD; the interaction matrix is trained via a stochastic feedforward network objective so that the basin equals low-probability-density regions.
Key results
- With CIFAR-10 as ID, RecLag achieved the lowest average FPR95 across ResNet18 (5.85 vs 7.41 for MHE), ResNet34 (5.14 vs 6.41), and WRN40-2 (7.47 vs 9.70 for MHE, though SHE was 10.00 there is also a lower-average claim as reported).
- With CIFAR-100 as ID (WRN40-2), RecLag reached average FPR95 35.29 versus 35.61 for SHE and 35.70 for MHE, a small margin.
- RecLag reported best AUC in 23 of 27 comparisons across the ROC curves.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RecLag instead of MHE | ResNet18 OOD detection · CIFAR-10 ID, nine OOD datasets (average) | FPR95 (%): 7.41 -> 5.85 | Helped |
| Use RecLag instead of MHE | ResNet34 OOD detection · CIFAR-10 ID, nine OOD datasets (average) | FPR95 (%): 6.41 -> 5.14 | Helped |
| Use RecLag instead of MHE | WRN40-2 OOD detection · CIFAR-10 ID, nine OOD datasets (average) | FPR95 (%): 9.70 -> 7.47 | Helped |
| Use RecLag instead of SHE | WRN40-2 OOD detection · CIFAR-100 ID, nine OOD datasets (average) | FPR95 (%): 35.61 -> 35.29 | Helped |
Surprising findings
- On CIFAR-100 with WRN40-2, RecLag's margin over the best Hopfield baseline is small (35.29 vs 35.61), and it is not best on Places or iNaturalist.
- Under the WRN40-2 CIFAR-10 setting, RecLag's average FPR95 of 7.47 is reported alongside a MHE average of 9.70 while SHE is listed at 10.00, so the table mixes bold markers inconsistently.
Limitations
- Only the memory-neuron Lagrangian is changed; the feature-neuron Lagrangian is kept as the simplest Euclidean form.
- Evaluated only in post hoc setting with frozen encoders; joint training with the classifier is left to future work.
- Performance on CIFAR-100 ID is only a small margin over baselines, and some OOD datasets (e.g., animals, people, foods) are hard to detect.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN LSUN-C LSUN-R iSUN Places365 DTD TinyImageNet SUN iNaturalist
Benchmarks: CIFAR-10 OOD benchmark (SVHN, LSUN-C, LSUN-R, iSUN, Places, DTD, TIN, SUN, iNaturalist) CIFAR-100 OOD benchmark
Metrics: FPR95 AUC
Baselines: MSP Energy ReAct MHE SHE
Builds on: modern Hopfield networks dense associative memory energy-based OOD detection stochastic feedforward neural network ReAct
Closest papers by meaning
- Continual Learning in Modern Hopfield Networks with an Application to Diffusion Models (NeurIPS 2026 · similarity 0.86)
- Relating Hopfield Networks to Episodic Control (NeurIPS 2024 · similarity 0.85)
- Outlier-Efficient Hopfield Layers for Large Transformer-Based Models (ICML 2024 · similarity 0.85)
- Energy-based Hopfield Boosting for Out-of-Distribution Detection (NeurIPS 2024 · similarity 0.85)
- Advancing Out-of-Distribution Detection via Local Neuroplasticity (ICLR 2025 · similarity 0.85)
- Sparse and Structured Hopfield Networks (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.