Research map / Vision-language and multimodal / Human and embodied modeling
L-WISE: Boosting Human Visual Category Learning Through Model-Based Image Selection and Enhancement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that logit-based difficulty scores and perturbations from robustified ANNs can predict and enhance human image recognition, and uses them in L-WISE to select and enhance training images, improving human category learning accuracy and reducing training time.
Problem
Human visual category learning is slow and hard for complex image domains (e.g., medical images); it is unclear whether robustified ANN models aligned with human vision can guide image curricula to improve human learning.
Approach
Use the ground-truth pre-softmax logit of a robustified ANN as a human-aligned difficulty score for selecting images, and maximize that logit via projected gradient ascent in pixel space to enhance category-specific features; combine both with easy-to-hard schedules (L-WISE) to train novice human learners.
Key results
- L-WISE increased test-phase accuracy margin above chance by 57.6% on the moth task, 72.3% on the dermoscopy task, and 33.1% on the histology task.
- L-WISE reduced training duration by 20% (moth), 23% (dermoscopy), and 22% (histology) with a fixed number of trials.
- Ground-truth logit of robustified ResNet-50 predicted human correctness with AUC 0.72 on 16-way ImageNet animal task, outperforming c-score and prediction depth per the authors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply L-WISE image selection and enhancement during training | Human learners · 4-way moth species classification | test accuracy margin above chance: +57.6% | Helped |
| Apply L-WISE image selection and enhancement during training | Human learners · 4-way skin lesion dermoscopy | test accuracy margin above chance: +72.3% | Helped |
| Apply L-WISE image selection and enhancement during training | Human learners · Binary colon histology | test accuracy margin above chance: +33.1% | Helped |
| Use L-WISE training assistance | Human learners · Moth species classification | training duration: 20% | Helped |
| Use L-WISE training assistance | Human learners · Skin lesion dermoscopy | training duration: 23% | Helped |
| Use L-WISE training assistance | Human learners · Colon histology | training duration: 22% | Helped |
| Add enhancement tapering to training (ablation) | Human learners · Moth species classification | test accuracy margin above chance: +46.8% | Helped |
| Add enhancement tapering to training (ablation) | Human learners · Skin lesion dermoscopy | test accuracy margin above chance: +56.5% | Helped |
| Use enhancement tapering without easy-to-hard ordering (ablation) | Human learners · Moth species classification | test accuracy margin above chance: +23.0% | Helped |
| Use enhancement tapering without easy-to-hard ordering (ablation) | Human learners · Skin lesion dermoscopy | test accuracy margin above chance: +11.2% | No clear effect |
| Add difficulty-based image selection (ablation) | Human learners · Moth species classification | test accuracy margin above chance: +8.1% | No clear effect |
| Add difficulty-based image selection (ablation) | Human learners · Skin lesion dermoscopy | test accuracy margin above chance: +53.2% | Helped |
Surprising findings
- Difficulty selection with shuffled trial order performed as well as or better than unshuffled, suggesting the benefit stems from easier image distribution rather than curriculum order.
- Combining enhancement and selection did not yield large additive accuracy gains over either alone, though additive training-speed gains were observed.
Limitations
- Proof-of-concept only; curriculum schedules and enhancement hyperparameters were not extensively optimized.
- Logit maximization can homogenize image distributions (e.g., benign moles look like smooth blobs), risking bias.
- Possible demographic and skin-tone bias in dermoscopy data raises concerns for clinical deployment.
Experimental setup
Datasets: ImageNet-1K iNaturalist 2021 HAM10000 MHIST Idaea moth photos from iNaturalist
Benchmarks: 16-way ImageNet animal categorization 4-way Idaea moth classification 4-way HAM10000 skin lesion classification binary MHIST colon histology classification
Metrics: test-phase accuracy accuracy margin above chance training duration AUC precision recall
Baselines: CLAHE MSRCR Adobe Lightroom Auto c-score prediction depth EXPLAIN CutMix ResNet-50 vanilla ResNet-50
Builds on: Robustness library adversarially trained ResNet-50 c-score prediction depth perceptually aligned gradients STRICT EXPLAIN
Closest papers by meaning
- Harmonizing the object recognition strategies of deep neural networks with humans (NeurIPS 2022 · similarity 0.89)
- Vision CNNs trained to estimate spatial latents learned similar ventral-stream-aligned representations (ICLR 2025 · similarity 0.87)
- Intriguing Properties of Generative Classifiers (ICLR 2024 · similarity 0.86)
- Evaluating alignment between humans and neural network representations in image-based learning tasks (NeurIPS 2024 · similarity 0.86)
- Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot (NeurIPS 2026 · similarity 0.86)
- Scaling Laws for Task-Optimized Models of the Primate Visual Ventral Stream (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.