AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Seq vs Seq: An Open Suite of Paired Encoders and Decoders

Orion Weller, Kathryn Ricci, Marc Marone, Antoine Chaffin, Dawn Lawrie, Benjamin Van Durme

ICLR 2026 · Large language models › Language, safety and interpretability · approach: LLM unlearning and benchmarks · 58 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use the same recipe for encoder as ModernBERT with open data (Ettin-Enc) instead of ModernBERTEttin-Enc-150m · GLUE average (base size)GLUE average: 88.4 -> 88.9Improved
Ettin decoder recipe versus Llama 3.2 1BEttin-Dec-1B · generative task average (1B)average accuracy: 56.6 -> 59.0Improved
continue pre-training a decoder with MNTP for 50B tokens (encoder-from-decoder) instead of using a pure encoder (ablation)Ettin-Dec-400m · MNLI (400M)accuracy: 91.3 (encoder) vs 87.6 (enc-from-dec)Hurt
add 50B tokens of MNTP continued training to a decoder for retrieval (ablation)Ettin-Dec-400m · MS MARCO dev (400M)nDCG@10: 39.93 -> 41.44Improved
pure encoder instead of encoder-from-decoder for retrieval (ablation)Ettin-Enc-from-Dec-400m · MS MARCO dev (400M)nDCG@10: 41.44 -> 42.24Improved
continue pre-training an encoder with CLM (decoder-from-encoder) instead of a pure decoder (ablation)Ettin-Enc-1B · generative average (1B)average accuracy: 59.0 (decoder) vs 52.2 (dec-from-enc)Hurt
model scale from 68m to 1B for decoder-from-encoder (ablation)decoder-from-encoder · generative tasksaverage accuracy: 42.1 vs 41.8 at 68m; 52.2 vs 59.0 at 1BHurt
use an encoder in generative fashion on classification-like tasks (ARC)Ettin-Enc-400m · ARC (400M)accuracy: 33.6 -> 35.6Improved
continue-train an encoder with CLM on a classification-style task (MMLU CS) (ablation)Ettin-Enc-1B · MMLU CS (1B)accuracy: 27.0 -> 37.0Improved
continue-train an encoder with CLM on a generative task (GSM8k) (ablation)Ettin-Enc-1B · GSM8k (1B)accuracy: 32.0 -> 18.9Hurt
use non-filtered data instead of filtered DCLM and higher-quality sources in later phases (ablation)Ettin training data · internal ablationdownstream performanceHurt
model merging (not used by Ettin)Ettin encoders · downstream applicationsperformanceImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy