Research map / Vision-language and multimodal / Human and embodied modeling
Sparse Multi-Modal Graph Transformer With Shared-Context Processing for Representation Learning of Giga-Pixel Images
Ramin Nakhli, Puria Azadi Moghadam, Haoyang Mi, Hossein Farahani, Alexander Baras, Blake Gilks, Ali Bashashati
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AMIGO instead of the strongest listed baseline | survival prediction · InUIT | C-Index: 0.55 ± 0.004 -> 0.57 ± 0.002 | Improved |
| Use AMIGO instead of the strongest listed baseline | survival prediction · MIBC | C-Index: 0.59 ± 0.007 -> 0.61 ± 0.004 | Improved |
| Add instance normalization after instance attention (ablation) | AMIGO · InUIT and MIBC | C-Index: InUIT: 0.53 ± 0.001 -> 0.57 ± 0.002; MIBC: 0.54 ± 0.005 -> 0.61 ± 0.004 | Improved |
| Share low-level modality branch weights while retaining modality-specific processing (ablation) | AMIGO · InUIT and MIBC | C-Index: InUIT: 0.56 ± 0.001 -> 0.57 ± 0.002; MIBC: 0.54 ± 0.016 -> 0.61 ± 0.004 | Improved |
| Keep modality branch weights partly shared rather than fully shared (ablation) | AMIGO · InUIT and MIBC | C-Index: InUIT: 0.53 ± 0.001 -> 0.57 ± 0.002; MIBC: 0.51 ± 0.005 -> 0.61 ± 0.004 | Improved |
| Use batch censored portion sampling (ablation) | AMIGO · InUIT and MIBC | C-Index: InUIT: 0.54 ± 0.001 -> 0.57 ± 0.002; MIBC: 0.58 ± 0.013 -> 0.61 ± 0.004 | Improved |
| Use shared instance attention rather than Transformer attention for within-modality aggregation (ablation) | AMIGO · InUIT and MIBC | C-Index: InUIT: 0.53 ± 0.002 -> 0.57 ± 0.002; MIBC: 0.55 ± 0.011 -> 0.61 ± 0.004 | Improved |
| Apply sparsity during training but not at inference (ablation) | AMIGO · InUIT and MIBC | C-Index: InUIT: 0.56 ± 0.001 -> 0.57 ± 0.002; MIBC: 0.55 ± 0.004 -> 0.61 ± 0.004 | Improved |
| Share instance attention across modalities (ablation) | AMIGO · InUIT and MIBC | C-Index: InUIT: 0.54 ± 0.001 -> 0.57 ± 0.002; MIBC: 0.58 ± 0.003 -> 0.61 ± 0.004 | Improved |
| Increase graph sparsity during training | AMIGO · InUIT | FLOPs: from 3.5 to 0.7 GigaFlops | Improved |
| Train with sparse graph inputs | AMIGO · InUIT | C-Index | No clear effect |
Closest papers by meaning
- Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis (NeurIPS 2024 · similarity 0.89)
- Transformer-Based Video-Structure Multi-Instance Learning for Whole Slide Image Classification (AAAI 2024 · similarity 0.88)
- Dynamic Entity-Masked Graph Diffusion Model for Histopathology Image Representation Learning (AAAI 2025 · similarity 0.88)
- Dynamic Graph Representation with Knowledge-aware Attention for Histopathology Whole Slide Image Analysis (CVPR 2024 · similarity 0.87)
- STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics (NeurIPS 2024 · similarity 0.87)
- MERGE: Multi-faceted Hierarchical Graph-based GNN for Gene Expression Prediction from Whole Slide Histopathology Images (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
