Research map / Label-efficient and robust learning / Recognition and long-tail
Contrastive Learning with Simplicial Convolutional Networks for Short-Text Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes C-SCN, a simplicial-complex message-passing network over document simplicial complexes (0/1/2-simplexes) contrasted with a BERT sequential encoder for few-shot short-text classification.
Problem
Short-text classification with few labels suffers from sparse words and semantics; existing contrastive and graph methods rely on augmentations that distort meaning, add noisy auxiliary information, and capture only pairwise relations.
Approach
Build a document simplicial complex from text (tokens as 0-simplexes, ordered edges as 1-simplexes, filled triangles as 2-simplexes), run a simplicial convolutional network over it, and contrast its structural representation with a transformer's sequential representation instead of using hand-crafted augmentations.
Key results
- C-SCN reaches 75.61% test accuracy on Twitter versus 73.16% for GIFT, the previous best there.
- C-SCN achieves 84.15 F1 on StackOverflow versus 83.07 for GIFT, and 69.46 F1 on MR versus 65.21 for GIFT.
- Ablations show removing contrastive learning and simplexes lowers accuracy on all four datasets (e.g., C-SCN minus 0-simplex gives 74.78 on Twitter vs 76.09 full).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use C-SCN instead of GIFT | short-text classification · Twitter | accuracy: 73.16 -> 76.09 | Helped |
| use C-SCN instead of GIFT | short-text classification · MR | accuracy: 65.16 -> 69.87 | Helped |
| use C-SCN instead of GIFT | short-text classification · Snippets | accuracy: 82.35 -> 85.56 | Helped |
| use C-SCN instead of GIFT | short-text classification · StackOverflow | accuracy: 82.94 -> 83.87 | Helped |
| add 1-simplexes and 2-simplexes (ablation) | C-SCN · Twitter | accuracy: 74.78 -> 76.09 | Helped |
| add 1-simplexes and 2-simplexes (ablation) | C-SCN · MR | accuracy: 68.27 -> 69.87 | Helped |
| add 1-simplexes and 2-simplexes (ablation) | C-SCN · Snippets | accuracy: 85.13 -> 85.56 | Helped |
| add 1-simplexes and 2-simplexes (ablation) | C-SCN · StackOverflow | accuracy: 82.36 -> 83.87 | Helped |
| add 2-simplexes (ablation) | C-SCN with 1-simplexes · Twitter | accuracy: 75.41 -> 76.09 | Helped |
| add 2-simplexes (ablation) | C-SCN with 1-simplexes · MR | accuracy: 68.77 -> 69.87 | Helped |
| add 2-simplexes (ablation) | C-SCN with 1-simplexes · Snippets | accuracy: 85.32 -> 85.56 | Helped |
| add 2-simplexes (ablation) | C-SCN with 1-simplexes · StackOverflow | accuracy: 82.58 -> 83.87 | Helped |
Surprising findings
- Pretrained BERT and RoBERTa underperform many graph-based models in the few-shot setting, with BERT at 51.16% accuracy on Twitter.
- Setting the contrastive weight eta to 1 yields lower performance than no contrastive loss, per the sensitivity study.
Limitations
- Evaluated only on four short-text datasets in a few-shot setting with 20 labels per class.
- Contribution of contrastive weight eta is sensitive and grid-searched; no principled selection shown.
- Baseline numbers are taken from GIFT (Liu et al., 2024) rather than re-run in the same setup.
Experimental setup
Datasets: Twitter MR Snippets StackOverflow
Benchmarks: Twitter MR Snippets StackOverflow
Metrics: accuracy F1-score
Baselines: TF-IDF LDA PTE CNN LSTM BERT RoBERTa TL-GNN TextGCN TextING HyperGAT DADGNN
Builds on: BERT GloVe TextING Simplicial message passing (Bodnar et al.) GIFT
Closest papers by meaning
- A Simple Graph Contrastive Learning Framework for Short Text Classification (AAAI 2025 · similarity 0.92)
- Improved Graph Contrastive Learning for Short Text Classification (AAAI 2024 · similarity 0.90)
- Boosting Short Text Classification with Multi-Source Information Exploration and Dual-Level Contrastive Learning (AAAI 2025 · similarity 0.89)
- A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics (NeurIPS 2025 · similarity 0.84)
- ReGCL: Rethinking Message Passing in Graph Contrastive Learning (AAAI 2024 · similarity 0.83)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.