Research map / Architectures and efficiency / RNNs and normalization
Processing Megapixel Images with Deep Attention-Sampling Models
Angelos Katharopoulos, Francois Fleuret
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use attention sampling instead of processing all grid patches with Deep MIL | Colon cancer classification · Colon cancer | test error: 0.093 ± 0.004 -> 0.093 ± 0.014 | No clear effect |
| Use attention sampling instead of processing all grid patches with Deep MIL | Colon cancer classification · Colon cancer | time/sample: 48.5 ms -> 1.8 ms | Improved |
| Use attention sampling instead of processing all grid patches with Deep MIL | Colon cancer classification · Colon cancer | memory/sample: 644 MB -> 21 MB | Improved |
| Use attention sampling instead of processing all grid patches with Deep MIL | Speed limit sign classification · Speed limits | test error: 0.083 ± 0.006 -> 0.089 ± 0.002 | No clear effect |
| Use attention sampling instead of processing all grid patches with Deep MIL | Speed limit sign classification · Speed limits | time/sample: 97.2 ms -> 8.5 ms | Improved |
| Use attention sampling instead of processing all grid patches with Deep MIL | Speed limit sign classification · Speed limits | memory/sample: 1,497 MB -> 86 MB | Improved |
| Reduce sampled patches from 10 to 5 (ablation) | ATS · Speed limits | test error: 0.095 ± 0.008 -> 0.089 ± 0.002 | Improved |
| Remove the entropy regularizer (ablation) | ATS · Colon cancer | attention exploration | Hurt |
| Increase the entropy regularizer weight (ablation) | ATS · Colon cancer | attention exploration | Mixed |
| Vary the number of sampled patches during training (ablation) | ATS · Colon cancer | learned attention distribution | No clear effect |
Closest papers by meaning
- Emulating Self-attention with Convolution for Efficient Image Super-Resolution (ICCV 2025 · similarity 0.86)
- UltraPixel: Advancing Ultra High-Resolution Image Synthesis to New Peaks (NeurIPS 2024 · similarity 0.86)
- Bridging the Divide: Reconsidering Softmax and Linear Attention (NeurIPS 2024 · similarity 0.86)
- Seeing More with Less: Human-like Representations in Vision Models (CVPR 2025 · similarity 0.85)
- LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate (NeurIPS 2024 · similarity 0.85)
- Win-Win: Training High-Resolution Vision Transformers from Two Windows (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
