Research map / Architectures and efficiency / RNNs and normalization
Uncovering the Latent Potential of Deep Intermediate Representations
Arnesh Batra, Arush Gumber, Aniket Khandelwal, Jashn Khemani, Anubha Gupta
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add GeoReg to LOES during fine-tuning (ablation) | LOES (k=3) · Stanford Cars (trainable DINOv2-S/14) | top-1 accuracy (%): 81.9 -> 82.7 | Improved |
| add GeoReg to LOES when the encoder is frozen (ablation) | LOES · frozen encoder | accuracy | No clear effect |
| add GeoReg to trainable BERT-base fine-tuning (ablation) | BERT-base fine-tuning · TweetEval Emoji | validation accuracy | Improved |
| use LOES layer selection instead of last layer | CLIP-B/32 frozen · Stanford Cars | test accuracy (%): 50.53 -> 59.95 | Improved |
| use LOES layer selection instead of last layer | DeiT-B/16 frozen · Mini-ImageNet | test accuracy (%): 98.63 -> 98.44 | Hurt |
| use LOES instead of last layer | ModernBERT-B · MTEB average (excl. Toxic) | accuracy (%): 54.25 -> 64.89 | Improved |
| select layers with LOES instead of entropy, curvature or intrinsic-dimension selection | ModernBERT-B · MTEB average (excl. Toxic) | accuracy (%): 61.15 -> 64.89 | Improved |
| LOES selection instead of random 4 layers (ablation) | ModernBERT-B · AM-Scenario | accuracy (%): 77.47 -> 78.51 | Improved |
| LOES instead of greedy probe-accuracy selection with 5 epochs (ablation) | ModernBERT-B · MTOP | accuracy (%): 94.55 -> 94.48 | No clear effect |
| increase k beyond 3-4 selected layers (ablation) | LOES · MTOP (ModernBERT-base) | accuracy (%): 0.02 point gain from k=3 to k=5 | No clear effect |
| increase k from 4 to 10 (ablation) | LOES · MTOP (BERT-base) | accuracy (%): 0.69 point gain for 2.5x parameters | Improved |
| vary ridge regularizer lambda from 1e-5 to 1e-2 (ablation) | LOES (k=4) · MTOP / AM-Scenario (ModernBERT-base) | accuracy (%): MTOP 94.27 to 94.80 | No clear effect |
Closest papers by meaning
- Layer by Layer: Uncovering Hidden Representations in Language Models (ICML 2025 · similarity 0.89)
- Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry (ICML 2025 · similarity 0.88)
- Tracing the Representation Geometry of Language Models from Pretraining to Post-training (NeurIPS 2025 · similarity 0.88)
- Connecting Neural Models Latent Geometries with Relative Geodesic Representations (NeurIPS 2025 · similarity 0.88)
- Tracing Representation Progression: Analyzing and Enhancing Layer-Wise Similarity (ICLR 2025 · similarity 0.88)
- Latent Functional Maps: a spectral framework for representation alignment (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
