Research map / Learning theory and algorithms / Prediction and decision losses
Using Random Effects to Account for High-Cardinality Categorical Features and Repeated Measures in Deep Neural Networks
Giora Simchoni, Saharon Rosset
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the high-cardinality feature as a random effect via LMMNN instead of ignoring it | DNN · Airbnb | test MSE: 0.156 -> 0.142 | Improved |
| Use LMMNN instead of Ignore | DNN · Drugs | test MSE: 2.74 -> 2.66 | Improved |
| Use LMMNN instead of Ignore | CNN · CelebA noseY | test MSE: 1.64 -> 1.39 | Improved |
| Use LMMNN instead of Ignore | MLP · UKB PA | test MSE: 0.812 -> 0.809 | No clear effect |
| Use LMMNN instead of entity embeddings | CNN · CelebA noseX | test MSE: 3.6 -> 1.54 | Improved |
| Use LMMNN instead of MeNets | CNN · CelebA noseY | test MSE: 12.3 -> 1.39 | Improved |
| Use one-hot encoding of the categorical feature instead of ignoring it, at high cardinality | DNN · Simulated g(Z)=Z, q=10^4, sigma_b^2=0.1 | test MSE: 1.22 -> 1.54 | Hurt |
| Increase cardinality q from 10^2 to 10^4 for OHE at sigma_b^2=10 | OHE · Simulated g(Z)=Z | test MSE: 1.56 -> 2.34 | Hurt |
| Increase cardinality q from 10^2 to 10^4 for LMMNN at sigma_b^2=10 | LMMNN · Simulated g(Z)=Z | test MSE: 1.10 -> 1.32 | Mixed |
| Use LMMNN instead of lme4 in nonlinear setting | lme4 · Simulated g(Z)=Z | test MSE: 2.97 -> 1.17 | Improved |
| Use LMMNN in the g(Z)=ZW setting at high cardinality instead of Embeddings | Embeddings · Simulated g(Z)=ZW, q=10^4, sigma_b^2=1 | test MSE: 14.4 -> 3.43 | Improved |
| Use LMMNN with a second categorical feature instead of Embeddings | Embeddings · Simulated two categorical features | test MSE: 1.34 -> 1.13 | Improved |
Closest papers by meaning
- Integrating Random Effects in Deep Neural Networks (JMLR 2023 · similarity 0.85)
- H-Likelihood Approach to Deep Neural Networks with Temporal-Spatial Random Effects for High-Cardinality Categorical Features (ICML 2023 · similarity 0.85)
- Random features models: a way to study the success of naive imputation (ICML 2024 · similarity 0.83)
- How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Model (ICML 2024 · similarity 0.83)
- Dataset Distillation for Pre-Trained Self-Supervised Vision Models (NeurIPS 2025 · similarity 0.82)
- Learning from Neighbors: Category Extrapolation for Long-Tail Learning (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
