Research map / Label-efficient and robust learning / Recognition and long-tail
Just Cluster It: An Approach for Exploration in High-Dimensions using Clustering and Pre-Trained Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes "Just Cluster It", an exploration method that estimates pseudo-counts by episodic GMM clustering and global cosine-threshold cluster tables over random or pre-trained DINOv2 features, evaluated on VizDoom and Habitat.
Problem
Count-based exploration in high-dimensional 3-D environments fails because pixel changes between transitions are large but not salient, so pseudo-count density models need appropriate state aggregation.
Approach
Cluster representations at two levels: episodic Gaussian mixture clustering of random or pre-trained DINOv2 embeddings, then a global cluster table where episodic centers are matched by cosine similarity against threshold kappa to accumulate pseudo-counts for an inverse-square-root intrinsic reward.
Key results
- On Habitat Apartment-0, clustering with DINOv2 features reaches about 17K average visitation counts, above RECODE (under 15K) and RND (about 5K).
- On VizDoom MyWayHome, the method with random features converges at about 1.5M (Sparse) and 1.8M (VerySparse) steps, faster than RIDE and Curiosity which need at least 5M steps.
- Random features clustered by the method perform slightly better than DINO on VizDoom, but DINO is clearly better on the visually complex Habitat setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use random features instead of DINO features | Just Cluster It · VizDoom MyWayHome (Sparse and VerySparse) | convergence steps: around 1.7M -> around 1.5M; around 1.9M -> around 1.8M | Helped |
| Use DINO features instead of random features | Just Cluster It · Habitat-Replica Apartment-0 | visitation counts: 14K -> 17K | Helped |
| Use Just Cluster It instead of RECODE with DINO features | DINO-based exploration · VizDoom MyWayHome Sparse | convergence steps: 5M steps -> around 1.7M | Helped |
| Use DINO features with Just Cluster It instead of RECODE | Habitat-Replica Apartment-0 · Habitat-Replica Apartment-0 | visitation counts: slightly under 15K -> 17K | Helped |
| Use clustering-based exploration instead of prediction-error methods | VizDoom MyWayHome · VizDoom MyWayHome Sparse | convergence steps: at least 5M steps | Helped |
| Use DINO-based Just Cluster It instead of Random Network Distillation | Habitat-Replica Apartment-0 · Habitat-Replica Apartment-0 | visitation counts: around 5K -> 17K | Helped |
| Increase cosine similarity threshold κ so the global cluster table becomes more granular (ablation) | Just Cluster It · Habitat-Replica Apartment-0 | visitation counts | Hurt |
| Remove episodic clustering (ablation) | Just Cluster It with DINO features · Habitat-Replica Apartment-0 | visitation counts | Hurt |
| Remove episodic clustering (ablation) | Just Cluster It with random features · Habitat-Replica Apartment-0 | visitation counts | No clear effect |
| Remove episodic clustering (ablation) | Just Cluster It · VizDoom MyWayHome VerySparse | extrinsic reward | Hurt |
| Replace DINOv2 small with DINOv2 large (ablation) | Just Cluster It · Habitat-Replica Apartment-0 | visitation counts | No clear effect |
| Add random noise to observations (ablation) | Just Cluster It · Habitat-Replica Apartment-0 | visitation counts | No clear effect |
Surprising findings
- Random noise concatenated to observations slightly improved performance for both the proposed method and RECODE, suggested to have a regularizing effect on clustering.
- Random features clustered effectively in VizDoom performed slightly better than DINO features.
Limitations
- Evaluated only on singleton MDPs; extension to contextual MDPs is untested.
- Performance degrades when the global cluster table becomes too granular (high kappa).
Experimental setup
Datasets: VizDoom MyWayHome Habitat Replica Apartment-0
Benchmarks: VizDoom MyWayHome Sparse VizDoom MyWayHome VerySparse Habitat Replica Apartment-0 visitation counts
Metrics: extrinsic reward visitation counts
Baselines: RECODE E3B Random Network Distillation Curiosity-driven exploration RIDE
Builds on: RECODE Never Give Up E3B Random Network Distillation PPO
Closest papers by meaning
- Unlocking the Power of Representations in Long-term Novelty-based Exploration (ICLR 2024 · similarity 0.88)
- Latent World Models For Intrinsically Motivated Exploration (NeurIPS 2020 · similarity 0.87)
- Exploring the Edges of Latent State Clusters for Goal-Conditioned Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstraction (ICLR 2025 · similarity 0.85)
- StateSpaceDiffuser: Bringing Long Context to Diffusion World Models (NeurIPS 2025 · similarity 0.85)
- Rethinking Exploration in Reinforcement Learning with Effective Metric-Based Exploration Bonus (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.