Research map / Vision-language and multimodal / Human and embodied modeling
Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning
Iro Laina, Ruth Fong, Andrea Vedaldi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use hard negatives from the most similar other class instead of random negatives | Human learnability evaluation of SeLa and MoCo clusters · Self-supervised classes, purity groups | semantic coherence: SeLa: 71.8 -> 55.3 (purity (0.3, 0.4]); MoCo: 89.8 -> 56.8 (purity (0.3, 0.4]) | Hurt |
| Select class captions using both within-class and other-class captions | Class-level caption generator · Google search, all classes | Binary accuracy (%): 89.48 ± 0.07 -> 91.18 ± 0.07 | Improved |
| Drop the negative term from caption selection (ablation) | Class-level caption generator · Google search, all classes | Top-1 accuracy (%): 7.88 -> 7.64 | Hurt |
| Use ROUGE-L word-level distance instead of sentence-embedding distance (ablation) | Class-level caption generator · Google search, all classes | Top-1 accuracy (%): 7.88 -> 6.12 | Hurt |
| Use a random image as search query instead of a class-level caption | Class description evaluation · Google search, all classes | Top-1 accuracy (%): 7.88 -> 4.83 | Hurt |
| Use a representative image as search query instead of a class-level caption | Class description evaluation · Google search, all classes | Top-1 accuracy (%): 7.88 -> 4.05 | Hurt |
| Increase cluster purity | SeLa and MoCo clusters · Self-supervised classes, random negatives | semantic coherence: SeLa: 71.8 -> 98.8; MoCo: 89.8 -> 99.2 | Improved |
| Use automatic rather than human-written class descriptions | SeLa and MoCo clusters · Self-supervised classes | describability: SeLa purity (0.3, 0.4]: 84.2 -> 78.7; MoCo purity (0.3, 0.4]: 78.3 -> 83.7 | Mixed |
Closest papers by meaning
- Enhancing Pre-trained Representation Classifiability can Boost its Interpretability (ICLR 2025 · similarity 0.87)
- Measuring Per-Unit Interpretability at Scale Without Humans (NeurIPS 2024 · similarity 0.86)
- ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.86)
- Interpretable Measures of Conceptual Similarity by Complexity-Constrained Descriptive Auto-Encoding (CVPR 2024 · similarity 0.85)
- Restyling Unsupervised Concept Based Interpretable Networks with Generative Models (ICLR 2025 · similarity 0.85)
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
