Research map / Label-efficient and robust learning / Recognition and long-tail
Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes R2T, a regional Transformer that re-embeds offline instance features online inside MIL, so R2T-MIL (AB-MIL + R2T) reaches foundation-model-level performance from ResNet-50 features on pathology diagnosis, subtyping and survival tasks.
Problem
Conventional computational pathology MIL uses frozen offline instance features (ResNet-50 or foundation models) that are not fine-tuned for the downstream task, limiting performance; end-to-end training is infeasible due to memory costs of gigapixel WSIs with many patches.
Approach
Insert an online re-embedding module (Re-embedded Regional Transformer) between offline feature extraction and MIL aggregation; it applies regional self-attention over spatially ordered patch grids, cross-region attention over aggregated region representatives, and an Embedded Position Encoding Generator (1-D conv inside attention) for positional information.
Key results
- With ResNet-50 features, R2T-MIL (AB-MIL + R2T) reaches 97.32 AUC on CAMELYON-16 versus 97.30 for AB-MIL with PLIP features, matching foundation-model performance.
- R2T-MIL achieves C-index of 61.13% on BLCA, 67.19% on LUAD and 60.95% on LUSC, outperforming the second-best compared method by 2.23, 3.08 and 1.77 points.
- R2T improved AB-MIL on every MIL framework tested and gave the largest gains among re-embedding variants (e.g. +8.41% on LUAD) at lower training time than N-MSA local (6.5s vs 29.8s per epoch).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add R2T feature re-embedding to AB-MIL with ResNet-50 features (ablation) | AB-MIL · CAMELYON-16 | AUC: 94.52 -> 97.32 | Helped |
| Add R2T feature re-embedding to AB-MIL with ResNet-50 features (ablation) | AB-MIL · TCGA-NSCLC | AUC: 95.28 -> 96.40 | Helped |
| Add R2T feature re-embedding to AB-MIL with ResNet-50 features (ablation) | AB-MIL · TCGA-LUAD | C-index: 58.78 -> 67.19 | Helped |
| Use local rather than global N-MSA for re-embedding (ablation) | AB-MIL with ResNet-50 features · CAMELYON-16 | AUC: 96.20 -> 96.47 | Helped |
| Replace global N-MSA with R2T local re-embedding (ablation) | AB-MIL with ResNet-50 features · TCGA-LUAD | C-index: 63.99 -> 67.19 | Helped |
| Use EPEG position encoding (ablation) | R2T · CAMELYON-16 | AUC: 96.82 -> 97.32 | Helped |
| Use EPEG position encoding (ablation) | R2T · TCGA-NSCLC | AUC: 96.01 -> 96.40 | Helped |
| Use EPEG position encoding (ablation) | R2T · TCGA-LUAD | C-index: 65.45 -> 67.19 | Helped |
| Replace EPEG with PEG7×7 (ablation) | R2T · CAMELYON-16 | AUC: 96.82 -> 95.47 | Hurt |
| Replace EPEG with PPEG (ablation) | R2T · CAMELYON-16 | AUC: 96.82 -> 93.00 | Hurt |
| Add CR-MSA across regions (ablation) | R2T · TCGA-LUAD | C-index: 63.03 -> 67.19 | Helped |
| Add CR-MSA across regions (ablation) | R2T · CAMELYON-16 | AUC: 96.89 -> 97.32 | Helped |
Surprising findings
- Re-embedding ResNet-50 features with R2T surpasses the PLIP foundation model features on several tasks, suggesting online feature fine-tuning can substitute for costly foundation model pretraining.
- Conventional positional encodings such as PEG and PPEG did not reliably improve performance, while larger PEG kernels sometimes hurt results.
Limitations
- Gains reported mainly on AB-MIL-based setups; the re-embedding is end-to-end trained on frozen offline features, so full feature extractor fine-tuning is not performed.
- Local region partition assumes spatial ordering of instances within a bag, which may not hold for all WSI patch layouts.
- Some reported improvements have overlapping standard deviations across folds.
Experimental setup
Datasets: CAMELYON-16 TCGA-BRCA TCGA-NSCLC TCGA-LUAD TCGA-LUSC TCGA-BLCA
Benchmarks: CAMELYON-16 TCGA-BRCA TCGA-NSCLC TCGA-LUAD TCGA-LUSC TCGA-BLCA
Metrics: accuracy AUC F1-score C-index
Baselines: AB-MIL CLAM DSMIL TransMIL DTFD-MIL IBMIL MHIM-MIL
Builds on: AB-MIL TransMIL PEG CLAM PLIP
Closest papers by meaning
- Do Multiple Instance Learning Models Transfer? (ICML 2025 · similarity 0.89)
- Revisiting End-to-End Learning with Slide-level Supervision in Computational Pathology (NeurIPS 2025 · similarity 0.88)
- xMIL: Insightful Explanations for Multiple Instance Learning in Histopathology (NeurIPS 2024 · similarity 0.86)
- Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathology (NeurIPS 2026 · similarity 0.85)
- SI-MIL: Taming Deep MIL for Self-Interpretability in Gigapixel Histopathology (CVPR 2024 · similarity 0.85)
- Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.