AI Paper Atlas

Research map / Benchmarks / GLUE

GLUE leaderboard

35 papers at ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR report results on GLUE. Best Avg.: 90.1 (ETHER+, ICML 2024).

Numbers are as each paper reports them, read by AI from its results tables and not checked by hand. Papers can differ in setting (training data, model size, extra data); the topic column shows what each paper is about. Open the paper before you compare or quote a number.

GLUE · Avg.

19 papers, each with its best reported number for its own method; higher is better.

#Avg. ↑MethodPaperTopic
190.1ETHER+
DeBERTaV3-base
ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections
ICML 2024 · GLUE benchmark table
Pruning and efficient tuning
289.8FLoRA
DeBERTaV3-base
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning
ICLR 2025 · Table 3
Pruning and efficient tuning
389i-Code
DeBERTa V3 Base
i-Code: An Integrative and Composable Multimodal Learning Framework
AAAI 2023 · Table: Comparison to previous
Cross-modal learning
488.6SIFT
RoBERTa-Large
Sparse is Enough in Fine-tuning Pre-trained Large Language Models
ICML 2024 · Table 1
Recognition and long-tail
587.9AutoBERT-Zero-large
AutoBERT-Zero
AutoBERT-Zero: Evolving BERT Backbone from Scratch
AAAI 2022 · Small and large model comparis
Neural architecture search
687.2EA-RoBERTa-Large
RoBERTa-Large
Evolving Attention with Residual Convolutions
ICML 2021 · GLUE results table
Attention and transformers
787.14Sine LoRA k=4
RoBERTa V3
Efficient Learning with Sine-Activated Low-Rank Matrices
ICLR 2025 · Table 1
Pruning and efficient tuning
886.4ConvBERTbase, 4M updates
BERTbase
ConvBERT: Improving BERT with Span-based Dynamic Convolution
NeurIPS 2020 · GLUE test results table
Attention and transformers
985.6DeLoRA
RoBERTa-base
Decoupling Angles and Strength in Low-rank Adaptation
ICLR 2025 · Table: Comparisons of differen
Pruning and efficient tuning
1084.6E-MAGIC-ATAnalyzing and Mitigating Interference in Neural Architecture Search
ICML 2022 · Table: Results of pre-training
Neural architecture search
9 more papers on this leaderboard, the baselines reported on it and a CSV download are free with a scime account.
Sign in free to see all

GLUE · average score

12 papers, each with its best reported number for its own method; higher is better.

#average score ↑MethodPaperTopic
190.23SalientLoRA (rt=276)
DeBERTaV3-base
Unveiling LoRA Intrinsic Ranks via Salience Analysis
NeurIPS 2024 · Table 1
Pruning and efficient tuning
289.68CapaBoost-PAdapter
DeBERTaV3-base
Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuning
ICLR 2024 · GLUE DeBERTaV3-base results ta
Pruning and efficient tuning
389.28Control
RoBERTa-large
From Weight-Based to State-Based Fine-Tuning: Further Memory Reduction on LoRA with Parallel Control
ICML 2025 · Table 2
Pruning and efficient tuning
487.6PACE
RoBERTabase
PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization
NeurIPS 2024 · Table 5
Recognition and long-tail
586.62Compacter (n=4)
T5-Base
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers
NeurIPS 2021 · GLUE results table
Pruning and efficient tuning
686.38PRAC (rank=4)
RoBERTa
PRAC: Principal-Random Subspace for LLM Activation Compression and Memory-Efficient Training
ICML 2026 · Table 4
Pruning and efficient tuning
785.6MPT
T5-Base
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning
ICLR 2023 · Table 1
Recognition and long-tail
883.28Baseline (pre-trained, span replace, 15%)
T5 base (encoder-decoder)
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
JMLR 2020 · Table 1
Language, safety and interpretability
981.8Ours-AdaBERTSaliency-Aware Neural Architecture Search
NeurIPS 2022 · Table 6
Neural architecture search
1080.66EST
GPT2-base
Evolving Subnetwork Training for Large Language Models
ICML 2024 · Table: Main results of the exp
RNNs and normalization
2 more papers on this leaderboard, the baselines reported on it and a CSV download are free with a scime account.
Sign in free to see all

GLUE · Avg. (dev)

10 papers, each with its best reported number for its own method; higher is better.

#Avg. ↑MethodPaperTopic
189.5ELECTRA-1.75MELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
ICLR 2020 · Table 2
Attention and transformers
289.424-task CA-MTL
RoBERTa-LARGE
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data
ICLR 2021 · Appendix, Baselines and Other
Continual fine-tuning of LLMs
387.7FOLNet: jmc.atpLearning Language Representations with Logical Inductive Bias
ICLR 2023 · Analysis of FOLNet architectur
Attention and transformers
487.39TUPE-Large
BERT-Large
Rethinking Positional Encoding in Language Pre-training
ICLR 2021 · Appendix B table
Attention and transformers
586.7E-MAGIC-ATAnalyzing and Mitigating Interference in Neural Architecture Search
ICML 2022 · Table: Results of pre-training
Neural architecture search
686.4SegaBERT-large
BERT-large
Segatron: Segment-Aware Transformer for Language Modeling and Understanding
AAAI 2021 · Table 2
Attention and transformers
786.3AutoBERT-ZeroAutoBERT-Zero: Evolving BERT Backbone from Scratch
AAAI 2022 · GLUE development-set table
Neural architecture search
883.12SMYRF-BERT, 50% memory
BERT base
SMYRF - Efficient Attention using Asymmetric Clustering
NeurIPS 2020 · GLUE results table
Attention and transformers
981.5W-PCAW-PCA Based Gradient-Free Proxy for Efficient Search of Lightweight Language Models
ICLR 2025 · Table: Statistical significanc
Neural architecture search
1081.4Span-based dynamic convolution
ConvBERTsmall
ConvBERT: Improving BERT with Span-based Dynamic Convolution
NeurIPS 2020 · Comparison of ConvBERT with di
Attention and transformers

Not affiliated with any conference, journal or the benchmark's authors. Corrections: contact@latenticlabs.com.