Research map / Label-efficient and robust learning / Recognition and long-tail
Learning Coherent Representations: A Topological Approach to Interpretability
Sigurd Gaukstad, Melvin Vaupel, Valdemar Kargård Olsen, Erik Hermansen, Benjamin Adric Dunn
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add COH coherence loss | autoencoder · single rotated digit 6 | tuned features % (MRL>0.5): 63.3 -> 100.0 (vs L1) | Improved |
| add COH coherence loss | autoencoder · single rotated digit 6 | MSE: 0.010 -> 0.011 (L1 vs COH) | No clear effect |
| add COH coherence loss instead of L1 sparsity | autoencoder · single rotated digit 6 | sparsity (active % of features): 19.5 -> 33.2 | Mixed |
| add COH coherence loss | autoencoder · two circles toy | pure features %: 1.2 -> 90.2 (vs L1) | Improved |
| add COH coherence loss | autoencoder · two circles toy | locality (coherence epsilon): 4.62 -> 0.14 (L1 vs COH) | Improved |
| add L1 sparsity alone (ablation) | autoencoder · two circles toy | locality: 0.53 -> 4.62 (vanilla vs L1) | Hurt |
| add COH coherence loss | autoencoder · two rotated digits 3 and 7 | MRL and tuned score | Mixed |
| add L1 sparsity to COH (ablation) | autoencoder · two rotated digits 3 and 7 | class separation and angle tuning across seeds | Improved |
| add COH loss to non-negative Softplus token embedding (ablation) | BERT token embedding · WikiText-2 | features with >50% overlap (of 256): 1.0 -> 77.4 | Improved |
| add COH loss to non-negative Softplus token embedding (ablation) | BERT token embedding · WikiText-2 | Claude-judged interpretable features (of 256): 0.0 -> 87.6 | Improved |
| add COH loss to non-negative Softplus token embedding (ablation) | BERT token embedding · WikiText-2 | mean overlap: 0.22 -> 0.45 | Improved |
| add COH loss | BERT token embedding · WikiText-2 | model performance | No clear effect |
Closest papers by meaning
- Formalizing Consistency and Coherence of Representation Learning (NeurIPS 2022 · similarity 0.87)
- A Curvature Phase Transition Governs Coherence Penalty Efficiency Against Feature Absorption in SAEs (NeurIPS 2026 · similarity 0.86)
- Neurons as Detectors of Coherent Sets in Sensory Dynamics (NeurIPS 2025 · similarity 0.85)
- Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry (ICML 2025 · similarity 0.85)
- Latent Functional Maps: a spectral framework for representation alignment (NeurIPS 2024 · similarity 0.84)
- Platonic Representations in the Human Brain: Unsupervised Recovery of Universal Geometry (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
