Research map / Architectures and efficiency / Attention and transformers
Sequencer: Deep LSTM for Image Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add horizontal BiLSTM to vertical-only (plus channel fusion) (ablation) | Sequencer2D-S · ImageNet-1K val | top-1 accuracy: 75.6 -> 82.3 | Helped |
| add vertical BiLSTM to horizontal-only (plus channel fusion) (ablation) | Sequencer2D-S · ImageNet-1K val | top-1 accuracy: 75.0 -> 82.3 | Helped |
| add channel fusion layer (ablation) | Sequencer2D-S · ImageNet-1K val | top-1 accuracy: 81.6 -> 82.3 | Helped |
| use bidirectional instead of unidirectional LSTMs (ablation) | Sequencer2D-S · ImageNet-1K val | top-1 accuracy: 79.7 -> 82.3 | Helped |
| use 2D parallel vertical/horizontal BiLSTM instead of vanilla 1D BiLSTM (ablation) | Sequencer-S · ImageNet-1K val | top-1 accuracy: 78.8 -> 82.3 | Helped |
| add learned positional embedding to vanilla Sequencer (ablation) | VSequencer-S · ImageNet-1K val | top-1 accuracy: 78.0 -> 78.1 | No clear effect |
| double LSTM hidden dimension (ablation) | Sequencer2D-S · ImageNet-1K val | top-1 accuracy: 82.3 -> 82.6 | No clear effect |
| replace LSTM with GRU (ablation) | Sequencer2D-S · ImageNet-1K val | top-1 accuracy: 82.3 -> 82.3 | No clear effect |
| replace LSTM with tanh-RNN (ablation) | Sequencer2D-S · ImageNet-1K val | top-1 accuracy: 82.3 -> 80.6 | Hurt |
| merge vertical/horizontal outputs by add instead of concatenate (ablation) | Sequencer2D-S · ImageNet-1K val | top-1 accuracy: 82.3 -> 82.2 | No clear effect |
| widen Sequencer2D-L layers by 4/3 (ablation) | Sequencer2D-L · ImageNet-1K val | top-1 accuracy: 83.4 -> 83.0 | Hurt |
Closest papers by meaning
- Convolutional Sequence to Sequence Learning (ICML 2017 · similarity 0.89)
- Vision-LSTM: xLSTM as Generic Vision Backbone (ICLR 2025 · similarity 0.88)
- LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognition (AAAI 2024 · similarity 0.86)
- Multiscale Low-Frequency Memory Network for Improved Feature Extraction in Convolutional Neural Networks (AAAI 2024 · similarity 0.85)
- Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.85)
- LaVin-DiT: Large Vision Diffusion Transformer (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.