Research map / Vision-language and multimodal / Human and embodied modeling
MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MICA, a concept-bottleneck skin lesion diagnosis framework that aligns images and clinical concepts at image, token, and concept-subspace levels, using a ClinicalBERT concept encoder and a ResNet-50 image encoder.
Problem
Concept-based explainable models for skin lesion diagnosis use concept annotations at only one level, which underuses medical information and lowers accuracy relative to black-box models when using concept bottlenecks.
Approach
Multi-level contrastive alignment between image and concept representations (global image-level, attention-based token-level with a frozen clinical LLM encoder, and CAV-based concept-subspace alignment) before a concept bottleneck diagnosis head, enabling concept intervention and visual/textual explanations.
Key results
- On Derm7pt, MICA (w/o bottleneck) reaches 85.59% AUC and 83.94% ACC for diagnosis, versus 83.27% AUC for the best baseline PCBM-h.
- On PH2, MICA (w/o bottleneck) achieves 98.18% AUC versus 97.50% for CBE.
- Ablation shows each of ILA, TLA, and CLA contributes; the full model reaches 84.1 AUC on Derm7pt diagnosis versus 71.2 without alignment modules.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MICA without a concept bottleneck instead of CBE | MICA · Derm7pt | AUC (%): 76.60$_{0.35}$ -> 85.59$_{1.11}$ | Helped |
| Use MICA without a concept bottleneck instead of CBE | MICA · PH² | AUC (%): 97.50$_{0.00}$ -> 98.18$_{1.43}$ | Helped |
| Use MICA without a concept bottleneck instead of CBE | MICA · SkinCon | AUC (%): 72.75$_{1.15}$ -> 75.92$_{1.13}$ | Helped |
| Add image-level alignment (ILA) (ablation) | MICA · Derm7pt; PH²; SkinCon | AUC_D (%): 82.9 -> 84.1; 96.1 -> 97.7; 75.0 -> 75.9 | Helped |
| Add token-level alignment (TLA) (ablation) | MICA · Derm7pt; PH²; SkinCon | AUC_D (%): 83.3 -> 84.1; 96.4 -> 97.7; 75.1 -> 75.9 | Helped |
| Add concept-level alignment (CLA) (ablation) | MICA · Derm7pt; PH²; SkinCon | AUC_D (%): 82.8 -> 84.1; 95.6 -> 97.7; 74.7 -> 75.9 | Helped |
| Add concept bottleneck (ablation) | MICA · Derm7pt | AUC (%): 85.59$_{1.11}$ -> 84.11$_{1.10}$ | Hurt |
Surprising findings
- Without the concept bottleneck, MICA's direct classification head outperforms the bottleneck version on diagnosis metrics.
Limitations
- Concept-level alignment relies on fixed LLM parameters and may carry inaccurate knowledge from the LLM.
- Performance drops notably on PH2 with only 10% of diagnosis labels (15 training labels).
- Evaluated only on three skin datasets.
Experimental setup
Datasets: Derm7pt PH2 SkinCon
Benchmarks: Derm7pt PH2 SkinCon
Metrics: AUC accuracy F1 concept detection accuracy label efficiency
Baselines: Sarkar et al. 2022 PCBM PCBM-h CBE CAV (Lucieri et al. 2020)
Builds on: Concept Bottleneck Model CLIP-style contrastive learning Concept Activation Vectors (TCAV) PCBM
Closest papers by meaning
- Towards Multi-dimensional Explanation Alignment for Medical Classification (NeurIPS 2024 · similarity 0.87)
- Interactive Medical Image Analysis with Concept-based Similarity Reasoning (CVPR 2025 · similarity 0.86)
- Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedback (NeurIPS 2025 · similarity 0.85)
- Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathology (NeurIPS 2026 · similarity 0.85)
- Decomposing Disease Descriptions for Enhanced Pathology Detection: A Multi-Aspect Vision-Language Pre-training Framework (CVPR 2024 · similarity 0.85)
- GRAPE: Graph-Augmented Prototype Explanations for Interactive Medical Image Diagnosis (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.