Research map / Label-efficient and robust learning / Recognition and long-tail
ReMasker: Imputing Tabular Data with Masked Autoencoding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReMasker, a masked-autoencoder Transformer for tabular data imputation that re-masks extra observed values during training and reconstructs them, evaluated on 12 UCI datasets against 13 imputers.
Problem
Imputing missing values in tabular data with high fidelity and utility, especially under high missingness ratios and limited data, where existing discriminative and generative imputers have limitations.
Approach
Extends masked autoencoding to tabular imputation: besides naturally missing values, randomly re-masks another subset of observed values during fitting and trains a Transformer autoencoder to reconstruct them; the trained model then predicts the missing entries. Theory suggests this encourages missingness-invariant representations.
Key results
- ReMasker outperforms all baseline imputers under at least one metric across all 12 datasets under MAR with 0.3 missingness.
- Under 0.7 missingness on the letter dataset, ReMasker still achieves RMSE below 0.1.
- Using ReMasker as the base imputer within HyperImpute improves letter RMSE from 0.0564 to 0.0554 and california RMSE from 0.0722 to 0.0702.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Transformer backbone instead of linear or convolutional (ablation) | ReMasker · letter and california (MAR 0.3) | RMSE: letter 0.0611 vs 0.1732 (linear), 0.1694 (conv) | Helped |
| Include reconstruction loss on unmasked values in addition to re-masked values (ablation) | ReMasker · california (MAR 0.3) | RMSE: 0.0663 vs 0.0840 (re-masked only) | Helped |
| Reconstruction loss on unmasked values only (no re-masked) (ablation) | ReMasker · letter (MAR 0.3) | RMSE: 0.2079 vs 0.0616 | Hurt |
| Use ReMasker as base imputer within HyperImpute instead of mean substitution (ablation) | HyperImpute · letter and california (MAR 0.3) | RMSE: letter 0.0564→0.0554; california 0.0722→0.0702 | Helped |
| Masking ratio increased from 0.5 to 0.7 (ablation) | ReMasker · letter and california (MAR 0.3) | RMSE: letter 0.0554→0.0906; california 0.0663→0.1320 | Hurt |
| Training epochs increased from 100 to 1600 (ablation) | ReMasker · letter (MAR 0.3) | RMSE | Helped |
| Decoder depth 8 vs other depths (default config) (ablation) | ReMasker · letter (MAR 0.3) | RMSE: best 0.0611 at depth 8 vs 0.0729 at depth 2 and 0.0673 at depth 10 | Mixed |
Surprising findings
- Including the reconstruction loss on unmasked values improves performance for tabular data, contrary to the vision MAE finding.
- Performance is generally better under MCAR than MAR or MNAR.
Limitations
- Performs better when re-masked and missing value distributions match; biased under MAR where re-masked values concentrate on observable features.
- MSE-trained objective favors per-value RMSE over distributional fidelity (WD), e.g., on the climate dataset.
- Unconditional reconstruction loss on unmasked values is needed; relying on re-masked values alone gives limited supervision.
Experimental setup
Datasets: California Housing Climate Model Simulation Crashes Concrete Strength Diabetes Estimation of Obesity Levels Credit Approval Wine Quality Raisin Spam Base Bike Sharing Demand Letter Recognition Yacht Hydrodynamics
Benchmarks: Letter Recognition California Housing 12 UCI datasets
Metrics: RMSE Wasserstein distance AUROC
Baselines: HyperImpute MIWAE EM GAIN ICE MICE MIRACLE MissForest
Builds on: masked autoencoder (MAE) HyperImpute MIWAE
Closest papers by meaning
- To Predict or Not to Predict? Proportionally Masked Autoencoders for Tabular Data Imputation (AAAI 2025 · similarity 0.93)
- CACTI: Leveraging Copy Masking and Contextual Information to Improve Tabular Data Imputation (ICML 2025 · similarity 0.93)
- Learning What Not to Impute: An Uncertainty-Aware Diffusion Framework for Meaningful Missingness (NeurIPS 2026 · similarity 0.87)
- TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation (ICLR 2025 · similarity 0.85)
- Hybrid Autoencoders for Tabular Data: Leveraging Model-Based Augmentation in Low-Label Settings (NeurIPS 2025 · similarity 0.85)
- Missing Data Imputation by Reducing Mutual Information with Rectified Flows (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.