Research map / Label-efficient and robust learning / Recognition and long-tail
Language Model Meets Prototypes: Towards Interpretable Text Classification Models through Prototypical Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A doctoral dissertation summary proposing interpretable text classification via prototype networks built on pretrained LM encoders, including a multi-head graph attention prototype network and planned GNN extension with contrastive learning.
Problem
Pretrained transformer language models are black boxes; interpretable prototype-based text classifiers have a performance gap relative to the original models, and graph prototypes for document classification are hard to make interpretable.
Approach
Replace heuristic similarity (e.g., cosine) between encoded inputs and prototypes with multi-head graph attention that learns edge weights between input representations and prototype nodes, so decisions are based on attention-derived relatedness; extend with sentiment incongruity loss for sarcasm and contrastive learning (GraphCL) for graph prototypes.
Key results
- The graph-attention prototype network achieves the best performance among prototype networks on five public benchmarks (binary, four-label, and ten-label classification), and is within 0.3% of or better than the original black-box models.
- Percentage of distinguished prototypes stays between 90% and 95% when the number of prototypes varies from 10 to 40.
- The sarcasm detection work (submitted to TACL) reports state-of-the-art results on benchmark datasets using sentiment incongruity loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the multi-head graph-attention prototype network instead of other prototype networks | Text classification with language-model encoders · Five public benchmark datasets | classification performance | Helped |
| Use the proposed prototype network instead of the original black-box model | Text classification with language-model encoders · Five public benchmark datasets | performance gap to original black-box models: within 0.3% | Mixed |
| Vary the number of prototypes from 10 to 40 | Multi-head Graph Attention-based prototype network · Training data | percentage of distinguished prototypes: 90%-95% | No clear effect |
Limitations
- The graph-based document classification extension is still in progress and its results are not reported.
- Sarcasm detection work is submitted (not accepted) and specifics are not given in the text.
- Only a high-level summary; datasets, LMs, and exact metrics are not named.
Experimental setup
Metrics: accuracy
Baselines: ProtoGNN attention-based prototype networks post-hoc explanation methods black-box LM classifiers
Builds on: prototypical networks graph attention network ProtoGNN GraphCL
Closest papers by meaning
- Concept Bottleneck Large Language Models (ICLR 2025 · similarity 0.85)
- Large Language Models are Interpretable Learners (ICLR 2025 · similarity 0.85)
- Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention (AAAI 2024 · similarity 0.84)
- Measuring and Guiding Monosemanticity (NeurIPS 2025 · similarity 0.84)
- Exploring Model Editing for LLM-based Aspect-Based Sentiment Classification (AAAI 2025 · similarity 0.84)
- Extracting Interpretable Task-Specific Circuits from Large Language Models for Faster Inference (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.