Research map / Architectures and efficiency / Attention and transformers
Contextformer: A Transformer with Spatio-Channel Attention for Context Modeling in Learned Image Compression
A. Burakhan Koyuncu, Han Gao, Atanas Boev, Georgii Gaikov, Elena Alshina, Eckehard Steinbach
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-channel attention by increasing channel segments N_cs (ablation) | Contextformer · Kodak | rate savings | Improved |
| use channel-first order instead of spatial-first order (ablation) | Contextformer · Kodak | BD-rate | Improved |
| use spatial-first order at low bitrate (bpp<0.3) (ablation) | Contextformer · Kodak | BD-rate | Mixed |
| use more than four channel segments (ablation) | Contextformer · Kodak | BD-rate | No clear effect |
| increase network depth beyond 8 layers (ablation) | Contextformer · Kodak | BD-rate | No clear effect |
| increase depth L and embedding size d_e together (ablation) | Contextformer · Kodak | BD-rate | Improved |
| use BDS instead of DS in encoder (ablation) | Contextformer · Kodak encoding time | time (s): 56 -> 32 | Improved |
| use BDS and SCS instead of DS in encoder (ablation) | Contextformer · 4K encoding time | time (s): 1240 -> 120 | Improved |
| use wavefront coding instead of DS in decoder (ablation) | Contextformer · Kodak decoding time | time (s): 62 -> 44 | Improved |
| use wavefront coding instead of DS in decoder (ablation) | Contextformer · 4K decoding time | time (s): 1440 -> 820 | Improved |
| replace Cui et al. context model with Contextformer at similar size | Cui et al. model · Kodak | BD-rate: 10.1% | Improved |
| replace Minnen&Singh with Contextformer | Minnen&Singh · Kodak | BD-rate vs VTM 16.2: 3.7% -> 6.9% | Improved |
Later papers that compared with this method
1 later accepted paper reports Contextformer as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- MambaIC: State Space Models for High-Performance Learned Image Compression (CVPR 2025 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Linear Attention Modeling for Learned Image Compression (CVPR 2025 · similarity 0.88)
- MambaIC: State Space Models for High-Performance Learned Image Compression (CVPR 2025 · similarity 0.88)
- Diversify, Contextualize, and Adapt: Efficient Entropy Modeling for Neural Image Codec (NeurIPS 2024 · similarity 0.87)
- FLAVC: Learned Video Compression with Feature Level Attention (CVPR 2025 · similarity 0.87)
- Causal Context Adjustment Loss for Learned Image Compression (NeurIPS 2024 · similarity 0.86)
- GLIC: General Format Learned Image Compression (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
