AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

Using Random Effects to Account for High-Cardinality Categorical Features and Repeated Measures in Deep Neural Networks

Giora Simchoni, Saharon Rosset

NeurIPS 2021 · Learning theory and algorithms › Prediction and decision losses · approach: Theory of neural network sensitivity · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add the high-cardinality feature as a random effect via LMMNN instead of ignoring itDNN · Airbnbtest MSE: 0.156 -> 0.142Improved
Use LMMNN instead of IgnoreDNN · Drugstest MSE: 2.74 -> 2.66Improved
Use LMMNN instead of IgnoreCNN · CelebA noseYtest MSE: 1.64 -> 1.39Improved
Use LMMNN instead of IgnoreMLP · UKB PAtest MSE: 0.812 -> 0.809No clear effect
Use LMMNN instead of entity embeddingsCNN · CelebA noseXtest MSE: 3.6 -> 1.54Improved
Use LMMNN instead of MeNetsCNN · CelebA noseYtest MSE: 12.3 -> 1.39Improved
Use one-hot encoding of the categorical feature instead of ignoring it, at high cardinalityDNN · Simulated g(Z)=Z, q=10^4, sigma_b^2=0.1test MSE: 1.22 -> 1.54Hurt
Increase cardinality q from 10^2 to 10^4 for OHE at sigma_b^2=10OHE · Simulated g(Z)=Ztest MSE: 1.56 -> 2.34Hurt
Increase cardinality q from 10^2 to 10^4 for LMMNN at sigma_b^2=10LMMNN · Simulated g(Z)=Ztest MSE: 1.10 -> 1.32Mixed
Use LMMNN instead of lme4 in nonlinear settinglme4 · Simulated g(Z)=Ztest MSE: 2.97 -> 1.17Improved
Use LMMNN in the g(Z)=ZW setting at high cardinality instead of EmbeddingsEmbeddings · Simulated g(Z)=ZW, q=10^4, sigma_b^2=1test MSE: 14.4 -> 3.43Improved
Use LMMNN with a second categorical feature instead of EmbeddingsEmbeddings · Simulated two categorical featurestest MSE: 1.34 -> 1.13Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy