AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Self-conditioning Pre-Trained Language Models

Xavier Suau Cuadros, Luca Zappella, Nicholas Apostoloff

ICML 2022 · Large language models › Language, safety and interpretability · approach: Controlled text generation · 1 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use expert-unit intervention instead of FUDGEGPT2-M gender parity generation · Occupational contexts, woman parityperplexity: 85.40, (83.80, 92.49) -> 69.50, (63.43, 71.72)Improved
Use expert-unit intervention instead of FUDGEGPT2-M gender parity generation · Occupational contexts, man parityperplexity: 83.29, (81.60, 129.09) -> 64.62, (63.54, 65.02)Improved
Use expert-unit intervention instead of PPLM-BoWGPT2-M gender parity generation · Occupational contexts, woman parityperplexity: 288.19,(202.16, 502.35) -> 69.50, (63.43, 71.72)Improved
Use expert-unit intervention instead of PPLM-BoWGPT2-M gender parity generation · Occupational contexts, man parityperplexity: 262.48, (153.82, 411.49) -> 64.62, (63.54, 65.02)Improved
Intervene on top-ranked expert units rather than lower-ranked or random units (ablation)Self-conditioning · Nurse and doctor contextsprobability of generating the target gender conceptImproved
Increase the number of intervened expert units (ablation)GPT2-L bird-conditioned generation · Bird conceptpresence of the conditioned conceptImproved
Set expert units to zero to remove a concept (ablation)Self-conditioning · Concept removalconcept removal effectivenessNo clear effect
Use more expert units for a weakly detected concept (ablation)GPT2-L self-conditioned generation · Work conceptgeneration qualityHurt
Use the self-conditioning intervention rather than PPLM-BoWGPT2 inference · Generation speedspeedup: 7.3×Improved
Use self-conditioning rather than FUDGE or PPLM-BoW at strong gender conditioningGender parity generation · Occupational contextsprobability of generating the literal conditioning wordImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy