Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Gen-Z, a zero-shot generative text classifier that scores p(x|label description) with contextualized, paraphrase-aggregated label descriptions, outperforming zero-shot baselines and matching few-shot ICL across open LMs.
Problem
Discriminative LM prompting for text classification is miscalibrated and brittle to prompt/verbalizer variations, and few-shot ICL is sensitive to demonstration choice and costly; zero-shot methods lack a way to incorporate contextual information.
Approach
Classify by generative likelihood p(x | z(y, context)) where label descriptions encode contextual variables (data source, domain, subject, author/reader attributes), and sum likelihoods over ~10 paraphrased descriptions per label to reduce prompt variance.
Key results
- Gen-Z with GPT-J achieves 91.7 macro-F1 on SST2 and 87.0 on MR zero-shot, beating the best zero-shot baselines on most datasets.
- Gen-Z zero-shot outperforms best 8/16-shot ICL baselines on several tasks, e.g., Ethos (Religion) 70.1 vs 67.9 for ICL-Gen and HS18 62.6 vs 57.3 for ICL-DC; it lags on DBpedia (80.1 vs 92.4 for ICL-DC).
- Gen-Z shows lower variance across prompt paraphrases than few-shot baselines, which vary by up to 9.9 macro-F1 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use contextual label descriptions with multiple paraphrases instead of one label description (ablation) | Gen-Z · Domain-aware classification tasks | macro-F1 | Helped |
| Remove domain or data-source information from label descriptions (ablation) | Gen-Z · Domain-aware classification tasks | macro-F1 | Hurt |
| Increase the number of paraphrased label descriptions (ablation) | Gen-Z · Domain-aware classification tasks | macro-F1 | Helped |
| Aggregate label-description probabilities with arithmetic mean rather than geometric or harmonic mean (ablation) | Gen-Z · Domain-aware classification tasks | macro-F1 | Helped |
| Use harmonic mean aggregation instead of arithmetic mean (ablation) | Gen-Z · Domain-aware classification tasks | macro-F1 | Mixed |
| Use author and addressee gender in label descriptions (ablation) | Gen-Z · Empowerment | F1: 81.1 -> 85.0 | Helped |
| Use age and education in label descriptions (ablation) | Gen-Z · Politeness | accuracy: 80.1 -> 83.3 | Helped |
| Use Gen-Z rather than the strongest zero-shot baseline | GPT-J (6B) · SST2 | macro-F1: 87.1 -> 91.7 | Helped |
| Use Gen-Z rather than the strongest zero-shot baseline | GPT-J (6B) · Tweet3 | macro-F1: 43.0 -> 41.1 | Hurt |
| Use Gen-Z rather than the strongest zero-shot baseline | GPT-J (6B) · Ethos (NO) | macro-F1: 63.2 -> 56.3 | Hurt |
| Use Gen-Z rather than the strongest zero-shot baseline | GPT-J (6B) · Emotions | macro-F1: 37.8 -> 32.7 | Hurt |
| Use Gen-Z rather than the strongest 8/16-shot baseline | GPT-J (6B) · HS18 | macro-F1: 57.3 -> 62.6 | Helped |
Surprising findings
- Discriminative baselines show no consistent trend when adding context or multiple label descriptions, sometimes hurting performance.
- Llama 2 13B performs slightly worse than 7B overall, reversing the scaling trend seen in other families.
Limitations
- Experiments limited to English datasets.
- Only binary gender used for demographic attributes; Western-centric social categories.
- Did not evaluate 70B or closed-source models (e.g., GPT-3) due to compute cost; not evaluated on sentence-pair or QA tasks.
Experimental setup
Datasets: SST2 SST5 Yelp2 Yelp5 Poem Sentiment Financial Phrasebank MR CR Tweet3 AGNews DBpedia Hate_speech18
Benchmarks: SST2 SST5 Yelp2 Yelp5 MR CR Tweet3 Financial Phrasebank Poem Sentiment AGNews DBpedia Hate_speech18
Metrics: macro-F1 accuracy F1
Baselines: Disc-single-NC Disc-PMI Gen-Single-NC ICL-Disc ICL-Gen ICL-PMI ICL-DC Z-ICL
Builds on: in-context learning as Bayesian inference noisy channel models PMI calibration (Holtzman et al.) Z-ICL
Closest papers by meaning
- GenZSL: Generative Zero-Shot Learning Via Inductive Variational Autoencoder (ICML 2025 · similarity 0.86)
- Compositional Generalization for Multi-Label Text Classification: A Data-Augmentation Approach (AAAI 2024 · similarity 0.86)
- Descriptor and Word Soups: Overcoming the Parameter Efficiency Accuracy Tradeoff for Out-of-Distribution Few-shot Learning (CVPR 2024 · similarity 0.85)
- Zero-Shot Robustification of Zero-Shot Models (ICLR 2024 · similarity 0.85)
- Liberating Seen Classes: Boosting Few-Shot and Zero-Shot Text Classification via Anchor Generation and Classification Reframing (AAAI 2024 · similarity 0.85)
- Label Propagation for Zero-shot Classification with Vision-Language Models (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.