AI Paper Atlas

Research map / Benchmarks / text8

text8 leaderboard

21 papers at ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR report results on text8. Best BPC: 1.06 (SAC (head adaptive), NeurIPS 2020).

Numbers are as each paper reports them, read by AI from its results tables and not checked by hand. Papers can differ in setting (training data, model size, extra data); the topic column shows what each paper is about. Open the paper before you compare or quote a number.

text8 · BPC

21 papers, each with its best reported number for its own method; lower is better.

#BPC ↓MethodPaperTopic
11.06SAC (head adaptive)SAC: Accelerating and Structuring Self-Attention via Sparse Adaptive Connection
NeurIPS 2020 · Table 2
Attention and transformers
21.07CP-Transformer (14)
Transformer
Efficient Representation Learning via Adaptive Context Pooling
ICML 2022 · Table: BPC and model size
Attention and transformers
31.09Transformer-LS
44M
Long-Short Transformer: Efficient Transformers for Language and Vision
NeurIPS 2021 · Char-LM results table
Attention and transformers
41.13T64Character-Level Language Modeling with Deeper Self-Attention
AAAI 2019 · Table 1
Attention and transformers
51.13Ouroboros + Adam
Transformer-XL
Ouroboros: On Accelerating Training of Transformer-Based Language Models
NeurIPS 2019 · Test bpc/PPL table
Attention and transformers
61.24EDLM
MDLM
Energy-Based Diffusion Language Models for Text Generation
ICLR 2025 · Text8 BPC table
Fast diffusion sampling
71.249capMZU+DTMulti-Zone Unit for Recurrent Neural Networks
AAAI 2020 · Table 2
RNNs and normalization
81.26Transformer XL Complex-order 6L
Transformer XL 6L
Encoding word order in complex embeddings
ICLR 2020 · Language modeling results tabl
RNNs and normalization
91.27Large RHN, recurrence depth 10Recurrent Highway Networks
ICML 2017 · Table 4
RNNs and normalization
101.29LayerNorm HM-LSTMHierarchical Multiscale Recurrent Neural Networks
ICLR 2017 · Table 2
Molecules and proteins
11 more papers on this leaderboard, the baselines reported on it and a CSV download are free with a scime account.
Sign in free to see all

Not affiliated with any conference, journal or the benchmark's authors. Corrections: contact@latenticlabs.com.