Research map / Large language models / Language, safety and interpretability
VAST: The Valence-Assessing Semantics Test for Contextualizing Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Nullify 8 top PCs (ablation) | GPT-2 top-layer CWEs · Bellezza, bleached context | VAST (Pearson's ρ): .32 -> .76 | Helped |
| Nullify 8 top PCs (ablation) | GPT-2 top-layer CWEs · Bellezza, random context | VAST (Pearson's ρ): .17 -> .58 | Helped |
| Nullify 8 top PCs (ablation) | GPT-2 top-layer CWEs · Bellezza, aligned context | VAST (Pearson's ρ): .42 -> .85 | Helped |
| Nullify 8 top PCs (ablation) | GPT-2 top-layer CWEs · Bellezza, misaligned context | VAST (Pearson's ρ): .14 -> .54 | Helped |
| Use misaligned rather than aligned context | GPT-2 layer 11 CWEs · Contextual settings | VAST (Pearson's ρ): .77 -> .55 | Hurt |
| Represent multiply tokenized words using the later-layer encoding rather than layer-0 encoding | GPT-2 CWEs · Warriner, multiply tokenized words | VAST (Pearson's ρ) | Helped |
| Use layer 8 rather than layer 0 | GPT-2 singly tokenized words · Warriner | VAST (Pearson's ρ): .70 in layer 0; .46 in layer 8 for multiply tokenized words | Mixed |
| Use the last subtoken rather than the mean representation (ablation) | GPT-2 multiply tokenized words · Warriner | VAST (Pearson's ρ) | Helped |
| Nullify 2 PCs (ablation) | GPT-2 layer 8 CWEs · Bleached and aligned contexts | VAST (Pearson's ρ): bleached: .87; aligned: .90 | Helped |
| Subtract the mean and nullify 2 PCs (ablation) | GPT-2 layer 8 bleached CWEs · SimLex-999 | Spearman's ρ: .45 -> .50 | Helped |
| Subtract the mean and nullify 2 PCs (ablation) | GPT-2 layer 8 bleached CWEs · WordSim-353 | Spearman's ρ: .64 -> .66 | Helped |
| Nullify 8 PCs (ablation) | GPT-2 top-layer CWEs · WEAT bias tests | stereotype-congruent bias effect size: 8 of 10 | Mixed |
Closest papers by meaning
- Deep contextualized word representations (ICLR 2018 · similarity 0.90)
- Less is More: Local Intrinsic Dimensions of Contextual Language Models (NeurIPS 2025 · similarity 0.88)
- The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embedding (NeurIPS 2024 · similarity 0.87)
- MMTEB: Massive Multilingual Text Embedding Benchmark (ICLR 2025 · similarity 0.87)
- Exploring Activation Patterns of Parameters in Language Models (AAAI 2025 · similarity 0.87)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.