Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Efficient Nonparametric Tensor Decomposition for Binary and Count Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ENTED, a nonparametric Gaussian-process tensor decomposition for binary and count tensors that uses Pólya-Gamma augmentation for conjugate natural-gradient SVI and sparse orthogonal inducing points.
Problem
Tensor completion for binary and count data is hard because traditional tensor decompositions rely on Gaussian likelihoods and predefined multilinear structures (CP/Tucker), and GP-based tensor models cannot handle discrete observations or scale stochastically.
Approach
Replace multilinear contractions with a GP, use Pólya-Gamma augmentation to obtain conjugate updates for Bernoulli and negative binomial likelihoods, and combine sparse orthogonally decoupled inducing points (SOLVE) with stochastic natural-gradient variational inference.
Key results
- On binary tensor completion, ENTED achieves the best AUC on Digg, Enron and DBLP (e.g., 0.767 AUC at rank 10 on Digg vs. 0.737 for GPTF).
- On count tensor completion, ENTED obtains the lowest RMSE on JHU, Article and EMS across ranks, with NLL around 1.35 on Article versus 1.63 for VBGCP.
- ENTED converges faster than GPTF thanks to natural-gradient updates, and is faster per epoch than GPTF and GPTF-PG when many inducing points are used.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace GPTF's Probit binary likelihood with ENTED's PG-augmented Bernoulli model and SOLVE inference | GPTF · Binary tensor completion, Digg and DBLP | AUC: 0.737 $\pm$ 0.017 -> 0.767 $\pm$ 0.032; 0.954 $\pm$ 0.002 -> 0.962 $\pm$ 0.002 | Helped |
| Replace GPTF's Probit binary likelihood with ENTED's PG-augmented Bernoulli model and SOLVE inference | GPTF · Binary tensor completion, Enron | AUC: 0.950 $\pm$ 0.013 -> 0.950 $\pm$ 0.013 | No clear effect |
| Use ENTED's PG-augmented Bernoulli model and SOLVE inference instead of GPTF | GPTF · Binary tensor completion, Enron and DBLP | NLL: 0.349 $\pm$ 0.038 -> 0.309 $\pm$ 0.063; 0.339 $\pm$ 0.010 -> 0.250 $\pm$ 0.007 | Helped |
| Use ENTED's NB count model and nonlinear GP tensor structure instead of GPTF's Gaussian likelihood | GPTF · Count tensor completion, JHU, Article, and EMS | RMSE: 0.546 $\pm$ 0.014 -> 0.406 $\pm$ 0.042; 0.655 $\pm$ 0.027 -> 0.636 $\pm$ 0.024; 0.403 $\pm$ 0.060 -> 0.301 $\pm$ 0.045 | Helped |
| Use ENTED's NB count model instead of VBGCP | VBGCP · Count tensor completion, JHU, Article, and EMS | NLL: 69.0 $\pm$ 0.5 -> 3.67 $\pm$ 0.07; 1.63 $\pm$ 0.01 -> 1.35 $\pm$ 0.01; 22.5 $\pm$ 1.8 -> 2.91 $\pm$ 0.04 | Helped |
| Use ENTED instead of VBGCP | VBGCP · JHU, rank 10 | MAPE: 0.286 $\pm$ 0.024 -> 0.292 $\pm$ 0.037 | Hurt |
| Increase the inducing-point count (ablation) | GPTF · Article, rank 10 | RMSE | Hurt |
| Increase the inducing-point count (ablation) | GPTF-PG and ENTED · Article, rank 10 | RMSE | Helped |
| Use ENTED's decoupled sparse orthogonal inducing points instead of GPTF and GPTF-PG inference (ablation) | GPTF and GPTF-PG · Article, rank 10 | epoch computing time | Mixed |
Surprising findings
- CoSTCo, a nonlinear CNN-based tensor model, performs poorly on the sparse binary tensors, which the authors attribute to overfitting and to the probability tensor not being low-rank.
Limitations
- NB shape hyper-parameter zeta is fixed to 20 for all datasets rather than tuned or learned.
- Slower than GPTF and GPTF-PG when the number of inducing points is small because natural gradients are implemented manually.
Experimental setup
Datasets: Digg Enron DBLP JHU Covid-19 Article (DeskDrop) EMS Montgomery County
Benchmarks: Digg Enron DBLP JHU Article EMS
Metrics: AUC NLL RMSE MAPE
Baselines: GCP BCP SBTR GPTF CoSTCo NCPD BPCP VB-GCP MDTF GPTF-PG
Builds on: GPTF (Gaussian process tensor factorization) Pólya-Gamma augmentation SOLVE-GP (sparse orthogonal variational GP) sparse variational Gaussian process scalable Bayesian tensor ring (SBTR)
Closest papers by meaning
- Nonparametric Decomposition of Sparse Tensors (ICML 2021 · similarity 0.91)
- Nonparametric Sparse Tensor Factorization with Hierarchical Gamma Processes (ICML 2022 · similarity 0.90)
- Doubly Decomposing Nonparametric Tensor Regression (ICML 2016 · similarity 0.89)
- Functional Bayesian Tucker Decomposition for Continuous-indexed Tensor Data (ICLR 2024 · similarity 0.86)
- Fast Tensor Completion via Approximate Richardson Iteration (ICML 2025 · similarity 0.86)
- High-Dimensional Tensor Regression With Oracle Properties (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.