Research map / Optimization / Implicit bias and generalization
Integrating Random Effects in Deep Neural Networks
Giora Simchoni, Saharon Rosset
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LMMNN random-effects NLL instead of one-hot encoding | DNN · Simulated single categorical feature, g(Z)=Z, sigma_b^2=10, q=10^4 | test MSE: 2.37 -> 1.29 | Improved |
| use LMMNN instead of entity embeddings | DNN · Simulated single categorical feature, g(Z)=Z, sigma_b^2=10, q=10^4 | test MSE: 2.12 -> 1.29 | Improved |
| use LMMNN instead of ignoring the categorical feature | DNN · Simulated single categorical feature, g(Z)=Z, sigma_b^2=10, q=10^4 | test MSE: 11.2 -> 1.29 | Improved |
| use LMMNN instead of MeNets | DNN · Simulated single categorical feature, g(Z)=Z, sigma_b^2=10, q=10^4 | test MSE: 2.80 -> 1.29 | Improved |
| use LMMNN instead of lme4 LMM | linear mixed model · Simulated single categorical feature, g(Z)=Z, sigma_b^2=0.1, q=10^2 | test MSE: 2.93 -> 1.10 | Improved |
| use LMMNN instead of lme4 when g(Z) is nonlinear and q is large | lme4 · Simulated single categorical feature, g(Z)=ZW, sigma_b^2=10, q=10^4 | test MSE: 13.8 -> 13.9 | No clear effect |
| use LMMNN instead of one-hot encoding with g(Z)=ZW | DNN · Simulated single categorical feature, g(Z)=ZW, sigma_b^2=10, q=10^4 | test MSE: 81.3 -> 13.9 | Improved |
| use LMMNN instead of OHE with three categorical features | DNN · Simulated 3 categorical features, g(Z)=Z, sigma_b^2=(0.3,0.3,0.3) | test MSE: 1.62 -> 1.16 | Improved |
| use LMMNN instead of LSTM for longitudinal data | DNN · Simulated longitudinal, Random mode, q=10^4 | test MSE: 1.40 -> 1.23 | Improved |
| use known RBF kernel random field (LMMNN-R) instead of learned embedding of locations (LMMNN-E) | LMMNN · Simulated spatial RBF, sigma_b0^2=10, sigma_b1^2=0.1, q=10^4 | test MSE: 2.92 -> 1.57 | Improved |
| use LMMNN-R instead of SVDKL | DNN · Simulated spatial RBF, sigma_b0^2=10, sigma_b1^2=0.1, q=10^4 | test MSE: 2.38 -> 1.57 | Improved |
| use LMMNN-R instead of SVDKL in typical spatial scenarios | DNN · Simulated spatial RBF, sigma_b0^2=0.1, sigma_b1^2=0.1, q=10^2 | test MSE: 1.09 -> 1.10 | No clear effect |
Closest papers by meaning
- H-Likelihood Approach to Deep Neural Networks with Temporal-Spatial Random Effects for High-Cardinality Categorical Features (ICML 2023 · similarity 0.88)
- Adversarially-Regularized Mixed Effects Deep Learning (ARMED) Models Improve Interpretability, Performance, and Generalization on Clustered (non-iid) Data (TPAMI 2023 · similarity 0.86)
- A theory of high dimensional regression with arbitrary correlations between input features and target functions: sample complexity, multiple descent curves and a hierarchy of phase transitions (ICML 2021 · similarity 0.86)
- On the Versatile Uses of Partial Distance Correlation in Deep Learning (ECCV 2022 · similarity 0.85)
- Mixed Sum-Product Networks: A Deep Architecture for Hybrid Domains (AAAI 2018 · similarity 0.85)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
