Research map / Large language models / Language, safety and interpretability
Neologism Learning for Controllability and Self-Verbalization
John Hewitt, Oyvind Tafjord, Robert Geirhos, Been Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| learn a neologism embedding (frozen model) for a simple concept | Gemma-3-4B-IT · 7 simple concepts (LIMA) | concept score increase (%): 92% of base-to-training gap closed on average | Improved |
| use single-word synonym self-verbalizations in place of the neologism (ablation) | Gemma-3-4B-IT · 7 simple concepts | concept score increase (%): 92% (neologism) -> 39% (best synonym), 25% (first synonym) | Mixed |
| use long questionnaire-based verbalization in place of the neologism (ablation) | Gemma-3-4B-IT · 7 simple concepts | concept score increase (%): 92% -> 83% average; use-like 103% -> 32% | Mixed |
| train with APO-up (DPO variant) instead of plain likelihood (ablation) | neologism learning · 7 simple concepts | concept score increase (%): 70% -> 92% | Improved |
| train with multiple prompt templates (ablation) | neologism learning · multi-template evaluation | concept score increase (%): 77% -> 86% | Improved |
| add hinge loss on embedding norm to multi-template training (ablation) | neologism learning · multi-template evaluation | concept score increase (%): 86% -> 89% | Improved |
| add hinge loss to multi-template training (ablation) | neologism learning · 2 concepts combined | concept score increase (%): 76% -> 54% | Hurt |
| learn neologisms instead of few-shot prompting for compositions | Gemma-3-4B-IT · Short+Numerical+Likely all three | goal score (harmonic mean): 0.387 -> 0.482 | Improved |
| learn neologism instead of few-shot for the complex Likely concept | Gemma-3-4B-IT · Likely only | success rate: 0.281 -> 0.667 | Improved |
| learn neologism instead of few-shot for simple Short concept | Gemma-3-4B-IT · Short only | goal score: 0.922 -> 0.736 | Hurt |
| in-context learning of the neologism with 10 examples instead of embedding training (ablation) | Gemma-3-4B-IT · 5 simple concepts | concept score increase (%): 46% average for in-context on Gemma | Hurt |
| learn neologisms for complex AxBench concepts | Gemma-3-4B-IT · AxBench (5 concepts) | overall score (0-2): 0.0-0.4 (no concept) -> 1.54-1.91 | Improved |
Closest papers by meaning
- Position: We Can’t Understand AI Using our Existing Vocabulary (ICML 2025 · similarity 0.89)
- Latent Principle Discovery for Language Model Self-Improvement (NeurIPS 2025 · similarity 0.85)
- Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond) (NeurIPS 2025 · similarity 0.84)
- Training Language Models to Explain Their Own Computations (NeurIPS 2026 · similarity 0.84)
- Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction (ICML 2025 · similarity 0.84)
- Improving Factuality and Reasoning in Language Models through Multiagent Debate (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
