Research map / Architectures and efficiency / Attention and transformers
Context Matters: Query-aware Dynamic Long Sequence Modeling of Gigapixel Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Querent, a query-aware sparse attention framework that uses min/max region metadata to select the top-K relevant regions per patch for dense attention on gigapixel WSIs, evaluated on classification and survival tasks.
Problem
Full self-attention over the tens of thousands of patches in a whole slide image is quadratically expensive, while existing linear or fixed local-global attention approximations sacrifice modeling power.
Approach
For each query patch, region-level min and max feature summaries are used to score regions by upper-bound interaction, and dense attention is computed only over the top-K selected regions, with a theorem claiming a bounded approximation of full attention.
Key results
- On survival prediction across 8 TCGA subsets, Querent reaches average C-Index 0.670 vs 0.647 for the second-best (WiKG and MambaMIL).
- On BCNB-ER, Querent achieves ACC 0.836, AUC 0.848, F1 0.739, outperforming all listed baselines.
- On UBC-OCEAN subtyping, Querent reports ACC 0.835 and AUC 0.956; random region selection gives ACC 0.746 in ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use min-max region metadata summarization instead of min-only summarization (ablation) | Querent · Region-pair distance relationship preservation | Pearson correlation: 0.937 -> 0.975 | Helped |
| Use min-max region metadata summarization instead of max-only summarization (ablation) | Querent · Region-pair distance relationship preservation | Pearson correlation: 0.959 -> 0.975 | Helped |
| Use min-max region metadata summarization instead of mean summarization (ablation) | Querent · Region-pair distance relationship preservation | Pearson correlation: 0.902 -> 0.975 | Helped |
| Use min-max region metadata summarization instead of mean-std summarization (ablation) | Querent · Region-pair distance relationship preservation | Pearson correlation: 0.897 -> 0.975 | Helped |
| Use min-max region metadata summarization instead of min-only summarization (ablation) | Querent · Region-pair distance relationship preservation | MSE: 0.012 -> 0.008 | Helped |
| Use min-max region metadata summarization instead of max-only summarization (ablation) | Querent · Region-pair distance relationship preservation | MSE: 0.018 -> 0.008 | Hurt |
| Use min-max region metadata summarization instead of mean summarization (ablation) | Querent · Region-pair distance relationship preservation | MSE: 0.058 -> 0.008 | Helped |
| Use min-max region metadata summarization instead of mean-std summarization (ablation) | Querent · Region-pair distance relationship preservation | MSE: 0.062 -> 0.008 | Helped |
| Use query-aware importance estimation instead of an estimation side network (ablation) | Querent · TCGA-LUAD TP53 | Accuracy: 0.580_{\pm 0.076} -> 0.678_{\pm 0.068} | Helped |
| Use query-aware importance estimation instead of random region selection (ablation) | Querent · TCGA-LUAD TP53 | Accuracy: 0.649_{\pm 0.063} -> 0.678_{\pm 0.068} | Helped |
| Use query-aware importance estimation instead of an estimation side network (ablation) | Querent · UBC-OCEAN | Accuracy: 0.731_{\pm 0.036} -> 0.835_{\pm 0.015} | Helped |
| Use query-aware importance estimation instead of random region selection (ablation) | Querent · UBC-OCEAN | Accuracy: 0.746_{\pm 0.056} -> 0.835_{\pm 0.015} | Helped |
Surprising findings
- Querent's min/max region-selection ablation on UBC-OCEAN gives only modest gains over random region selection on TCGA-LUAD TP53 (ACC 0.678 vs 0.649).
Limitations
- The theorem's assumptions (exponential attention decay with spatial distance, spatially separated regions, diameter bounds) are strong and not verified empirically for real WSIs.
- Reported gains on TCGA-LUAD TP53 are small relative to standard deviations.
Experimental setup
Datasets: BCNB TCGA-LUAD UBC-OCEAN TCGA-BRCA TCGA-UCEC TCGA-STAD TCGA-LUSC TCGA-SKCM TCGA-KIRC TCGA-KIRP
Benchmarks: BCNB-ER TCGA-LUAD TP53 UBC-OCEAN TCGA survival (8 subsets)
Metrics: accuracy AUC F1 score C-index
Baselines: Mean Pooling Max Pooling ABMIL DS-MIL DTFD WiKG MambaMIL TransMIL HIPT HistGen RRT-MIL LongMIL
Builds on: ABMIL TransMIL Nystromformer HIPT MambaMIL
Closest papers by meaning
- Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis (NeurIPS 2024 · similarity 0.89)
- SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding (CVPR 2025 · similarity 0.87)
- PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.87)
- Dynamic Graph Representation with Knowledge-aware Attention for Histopathology Whole Slide Image Analysis (CVPR 2024 · similarity 0.86)
- Fast and Accurate Gigapixel Pathological Image Classification with Hierarchical Distillation Multi-Instance Learning (CVPR 2025 · similarity 0.85)
- Promptable Representation Distribution Learning and Data Augmentation for Gigapixel Histopathology WSI Analysis (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.