Research map / Label-efficient and robust learning / Self-supervised and contrastive
Random Forest Autoencoders for Guided Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Random Forest Autoencoders (RF-AE), an autoencoder that reconstructs RF-GAP proximities and is regularized toward RF-PHATE embeddings, to enable out-of-sample supervised visualization.
Problem
Supervised manifold visualization methods such as RF-PHATE lack an explicit mapping, so they cannot embed unseen data efficiently or scale to large datasets and label-scarce settings.
Approach
Replace the input of a regularized autoencoder with row-normalized, symmetrized RF-GAP proximity vectors, reconstruct them with a Jensen-Shannon divergence loss, and regularize the 2D bottleneck toward precomputed RF-PHATE embeddings; class-wise k-medoids prototype selection reduces input dimensionality.
Key results
- RF-AE (lambda=0.01, N*=0.1N) achieves the highest mean k-NN accuracy (0.831 ± 0.011) across 20 datasets, versus 0.809 for SSNP and 0.794 for RF-PHATE.
- RF-AE ranks top two on local SIA (QNX 0.800, Trust 0.818) and top on global SIA (Spear 0.776, Pearson 0.776) among 14 methods.
- On a Sign MNIST example, RF-AE reaches local SIA 0.89 versus 0.62 for parametric t-SNE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add geometric regularization (lambda<1, toward RF-PHATE embedding) to RF-AE (ablation) | RF-AE · 20 datasets (10 reps) | Global SIA Spear: 0.701 at lambda=1 to 0.776 at lambda=0.01 | Helped |
| add geometric regularization (lambda<1) to RF-AE (ablation) | RF-AE · 20 datasets (10 reps) | k-NN accuracy: 0.837 at lambda=1 vs 0.831 at lambda=0.01 | Hurt |
| RF-AE with lambda=0.01 vs. default RF-PHATE linear kernel extension | RF-AE · 20 datasets (10 reps) | k-NN accuracy: 0.831 vs 0.794 | Helped |
| RF-AE (lambda=0.01) vs. standard supervised baselines on SIA metrics | RF-AE · 20 datasets (10 reps) | Local SIA QNX: 0.800 vs CE 0.795 and NCA 0.808 (NCA ranks first) | Mixed |
| lambda=0.001 vs lambda=0.01 in RF-AE (ablation) | RF-AE · 20 datasets (10 reps) | k-NN accuracy: 0.827 vs 0.831 | No clear effect |
Surprising findings
- Reconstructing RF-GAP proximities with JSD outperforms reconstructing raw features, and the feature-based MLP is more sensitive to noise than the kernel-based encoder in the artificial tree experiment.
- Parametric supervised UMAP and SSNP reach high k-NN accuracy yet show notable drops in structural importance alignment, indicating class separation at the expense of structure.
Limitations
- Scalability is still limited by the computational cost of computing RF-GAP proximities.
- Structural importance alignment relies on pseudo-ground-truth importances from a baseline classifier ensemble, since true feature importances are rarely available.
- Code is promised but no release link is given in the text.
Experimental setup
Datasets: Sign MNIST (A-K) MNIST Fashion MNIST GTZAN (3-sec) USPS Blood MNIST OrganC MNIST Isolet Landsat Satellite Optical Digits HAR Cardiotocography
Benchmarks: 20 datasets from UCI, Kaggle, Zenodo
Metrics: k-NN accuracy structural importance alignment (Kendall tau) with QNX, Trus
Baselines: RF-PHATE kernel extension vanilla AE PCA supervised PCA parametric t-SNE parametric UMAP parametric supervised UMAP PaCMAP Centroid Encoders CEBRA SSNP NCA
Builds on: RF-PHATE RF-GAP GRAE PHATE parametric t-SNE
Closest papers by meaning
- Autoencoding Random Forests (NeurIPS 2025 · similarity 0.86)
- Graph Geometry-Preserving Autoencoders (ICML 2024 · similarity 0.84)
- LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusion (ICLR 2025 · similarity 0.82)
- Neuro-Visualizer: A Novel Auto-Encoder-Based Loss Landscape Visualization Method With an Application in Knowledge-Guided Machine Learning (ICML 2024 · similarity 0.82)
- Mixed-curvature decision trees and random forests (ICML 2025 · similarity 0.82)
- Partition First, Embed Later: Laplacian-Based Feature Partitioning for Refined Embedding and Visualization of High-Dimensional Data (ICML 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.