Research map / Vision-language and multimodal / Human and embodied modeling
MEATRD: Multimodal Anomalous Tissue Region Detection Enhanced with Spatial Transcriptomics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MEATRD, a multimodal (histology image + spatial transcriptomics) anomalous tissue region detector that reconstructs inlier spots with a masked graph dual-attention transformer and then fits a one-class classifier on latent multimodal reconstruction errors.
Problem
Detecting anomalous tissue regions (ATRs) in tissue slices where ATRs look visually similar to normal tissue, using histology images and spatial transcriptomics, which previously lacked a method that fuses both modalities.
Approach
Treats spots as nodes in a k-NN attributed graph; a masked graph dual-attention transformer (MGDAT) with target-node masking and a fused bottleneck embedding reconstructs inlier image patches and gene profiles, then a one-class classifier (SVDD-style hypersphere) on latent multimodal reconstruction errors scores anomalies, with a MAP-EM GMM for thresholding.
Key results
- MEATRD achieves mean AUC 0.723 and F1 0.741 across eight breast cancer ST datasets, ranking first in AUC, versus second-best DOMINANT (AUC 0.550) and scmap (F1 0.672).
- Reported average gain over the second-best method of 17.45% in AUC and 10.31% in F1.
- On 10x-hBC-I1 (visually subtle tumor edge), MEATRD-beta reaches precision 0.693 and recall 0.895.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Integrate histology images and ST data with spatial-context modeling | MEATRD · Mean across eight 10x-hBC datasets | AUC: 0.550 -> 0.723 | Helped |
| Integrate histology images and ST data with spatial-context modeling | MEATRD · Mean across eight 10x-hBC datasets | F1: 0.577 -> 0.741 | Helped |
| Add histology images to ST-only MEATRD (ablation) | MEATRD · Mean across eight 10x-hBC datasets | AUC: 0.631 -> 0.723 | Helped |
| Add ST data to image-only MEATRD (ablation) | MEATRD · Mean across eight 10x-hBC datasets | AUC: 0.497 -> 0.723 | Helped |
| Add MGDAT (ablation) | MEATRD · Mean across eight 10x-hBC datasets | AUC: 0.639 -> 0.723 | Helped |
| Add target-node masking (ablation) | MEATRD · Mean across eight 10x-hBC datasets | AUC: 0.655 -> 0.723 | Helped |
| Use latent multimodal reconstruction errors instead of direct spot embeddings (ablation) | MEATRD · Mean across eight 10x-hBC datasets | AUC: 0.642 -> 0.723 | Helped |
| Add the Stage III one-class classifier (ablation) | MEATRD · Mean across eight 10x-hBC datasets | AUC: 0.584 -> 0.723 | Helped |
| Reduce image weight β from 0.5 to 0.1 (ablation) | MEATRD · 10x-hBC-I1 tumor-edge detection | precision: 3.1% | Helped |
| Reduce image weight β from 0.5 to 0.1 (ablation) | MEATRD · 10x-hBC-I1 tumor-edge detection | recall: 34.4% | Helped |
| Set α to 0.9 rather than 0.5 (ablation) | MEATRD · Mean across eight 10x-hBC datasets | AUC: 0.723 -> 0.678 | Hurt |
| Set bottleneck dimension to 256 rather than 16 (ablation) | MEATRD · Mean across eight 10x-hBC datasets | AUC: 0.723 -> 0.682 | Hurt |
Surprising findings
- Image-only ablation yields mean AUC 0.497, near chance, while ST-only reaches 0.631.
- On the subtle-edge dataset, DOMINANT marginally outperforms MEATRD-standard, suggesting histology adds little there.
Limitations
- Histology contributes little when ST data is high quality and ATRs are visually subtle (e.g., DOMINANT edges out MEATRD on one dataset).
- Relies on hyperparameters such as alpha, beta, and MGDAT depth that are sensitive (4 layers collapses AUC to 0.533).
Experimental setup
Datasets: 10x-hNB-v03 10x-hNB-v04 10x-hNB-v05 10x-hNB-v06 10x-hNB-v07 10x-hNB-v08 10x-hNB-v09 10x-hNB-v10 10x-hBC-A1 10x-hBC-B1 10x-hBC-C1 10x-hBC-D1
Benchmarks: 10x-hBC-A1 10x-hBC-B1 10x-hBC-C1 10x-hBC-D1 10x-hBC-E1 10x-hBC-F1 10x-hBC-G2 10x-hBC-H1 PSC datasets
Metrics: AUC F1
Baselines: M3DM SimpleNet f-AnoGAN Patch SVDD DOMINANT PREM Spatial-ID scmap CAMLU
Builds on: Patch SVDD DOMINANT M3DM Deep SVDD U-Net
Closest papers by meaning
- STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics (NeurIPS 2024 · similarity 0.89)
- HEST-1k: A Dataset For Spatial Transcriptomics and Histology Image Analysis (NeurIPS 2024 · similarity 0.89)
- M2OST: Many-to-one Regression for Predicting Spatial Transcriptomics from Digital Pathology Images (AAAI 2025 · similarity 0.88)
- Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images (ICLR 2025 · similarity 0.87)
- STINR: Deciphering Spatial Transcriptomics via Implicit Neural Representation (CVPR 2025 · similarity 0.86)
- ASIGN: An Anatomy-aware Spatial Imputation Graphic Network for 3D Spatial Transcriptomics (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.