Research map / Label-efficient and robust learning / Recognition and long-tail
Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SUFO, a framework of supervised probing, unsupervised similarity (RSA), feature dynamics (PCA), and outlier analysis to interpret fine-tuned transformer feature spaces, applied to prostate pathology report classification and validated on MedNLI.
Problem
Fine-tuned pre-trained transformers are used for clinical decision-making, but little is understood about their feature spaces, how they evolve during fine-tuning, and their failure modes, which limits trust and safety in medicine.
Approach
SUFO integrates supervised probing, unsupervised RSA, PCA-based feature dynamics across layers and epochs, and PCA-based cluster outlier extraction with clinician review into one pipeline for interpreting fine-tuned transformer features, used to study how pre-training data domain affects fine-tuned models.
Key results
- PubMedBERT achieves the highest average supervised probing F1 (0.494) but fine-tuned PubMedBERT drops to 0.770 F1 on Path-PG, with 0.67 per-class accuracy on the minority class 5, while Clinical BioBERT reaches 0.959 F1 on Path-PG.
- Clinical BioBERT has the smallest layer-wise RSA change after fine-tuning, while TNLR undergoes the most drastic reconfiguration.
- The first two principal components explain on average 95% of feature variance in fine-tuned last-layer CLS features; a domain expert found clinical-domain models (Clinical BioBERT, PubMedBERT) identify more cases with missing target information (type 4 outliers: 3 and 5 vs 0-1 for BERT/BioBERT).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use mixed-domain pretraining (BioBERT or Clinical BioBERT) instead of general-domain BERT | fine-tuned encoder classifier · Path-PG | F1: 0.858 -> 0.933; 0.858 -> 0.959 | Helped |
| Use PubMedBERT instead of Clinical BioBERT | fine-tuned encoder classifier · Path-PG | F1: 0.959 -> 0.770 | Hurt |
| Use PubMedBERT instead of Clinical BioBERT | fine-tuned encoder classifier · MedNLI Balanced | accuracy: 0.73 (0.01) -> 0.83 (0.01) | Helped |
| Use PubMedBERT instead of Clinical BioBERT | fine-tuned encoder classifier · MedNLI Imbalanced | accuracy: 0.71 (0.01) -> 0.70 (0.02) | Hurt |
| Use PubMedBERT instead of Clinical BioBERT | fine-tuned encoder classifier · MedNLI Highly Imbalanced | accuracy: 0.76 (0.03) -> 0.71 (0.01) | Hurt |
| Use PubMedBERT pretrained features rather than randomly initialized BERT features in supervised probing | linear-probe classifier · Path-PG | F1: 0.339 (0.06) -> 0.387 (0.01) | Helped |
| Use in-domain pretraining rather than general-domain pretraining | fine-tuned encoder classifier · Path-PG feature dynamics | epochs to begin class disambiguation: around epoch 9 -> around epoch 6 | Helped |
| Use Clinical BioBERT rather than PubMedBERT | fine-tuned encoder classifier · Path-PG | minority-class 5 accuracy: 0.67 -> 1.00 | Helped |
| Use in-domain pretraining (Clinical BioBERT or PubMedBERT) rather than general-domain pretraining for outlier detection | fine-tuned feature-space outlier analysis · Pathology reports, hard outlier type 4 | number of identified hard outliers: BERT 0; Clinical BioBERT 3; PubMedBERT 5 | Helped |
Surprising findings
- PubMedBERT, the best supervised-probing model, performs close to general-domain models after fine-tuning on the imbalanced Path-PG task, overfitting to the minority class.
- Random-BERT features reach lower probing scores than all pre-trained models on most tasks, but TNLR's probing average (0.378) falls below the random baseline (0.421).
Limitations
- Pathology report dataset is private and limited in scale; findings restricted to clinical classification tasks.
- Outlier analysis relies on a single domain expert (urology clinician) and uses the training set, limiting test-set generalization analysis.
- TNLR differs in pre-training objective and attention, which may confound comparisons.
Experimental setup
Datasets: Prostate Cancer Pathology Reports (UCSF, 2907 reports) MedNLI
Benchmarks: Prostate Cancer Pathology Reports (Path-PG, Path-SG, Path-MS, Path-SV) MedNLI
Metrics: macro F1 accuracy RSA Pearson correlation per-class accuracy
Baselines: Random-BERT
Builds on: BERT PubMedBERT BioBERT Clinical BioBERT TNLR
Closest papers by meaning
- RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis (NeurIPS 2025 · similarity 0.85)
- A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis (NeurIPS 2024 · similarity 0.84)
- CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMs (NeurIPS 2025 · similarity 0.84)
- Trade-Offs in Fine-Tuned Diffusion Models between Accuracy and Interpretability (AAAI 2024 · similarity 0.83)
- OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoning (CVPR 2026 · similarity 0.83)
- MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.