Research map / Graph learning / Graph self-supervised learning
Improved Graph Contrastive Learning for Short Text Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GIFT, a graph contrastive learning model for short text classification that uses SVD-reconstructed term-document matrices as augmented views and constrained seed k-means to drive cluster-oriented contrastive learning.
Problem
Short text classification suffers from semantic sparsity and scarce labeled data; existing graph contrastive methods generate augmented views by perturbation or word deletion that distort semantics, and instance-discrimination contrastive learning creates false negatives among semantically similar t
Approach
Build a heterogeneous corpus-level graph (word, entity, POS-tag component graphs), generate augmented views by truncated SVD of each text-specific term-document matrix, and use constrained seed k-means seeded by labeled texts to assign weak labels for cluster-oriented contrastive learning.
Key results
- GIFT reaches 73.16% ACC on Twitter, 65.21% on MR, 83.73% on Snippets and 83.07% on StackOverflow, the best among all compared methods.
- Compared with the strongest baseline SHINE, GIFT improves StackOverflow accuracy from 73.05% to 83.07%.
- Ablation shows removing SVD-based augmentation and cluster-oriented CL together drops StackOverflow ACC to 76.20%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SVD-based contrastive learning and cluster-oriented contrastive learning (ablation) | GIFT-SVD-K · Twitter, MR, Snippets, StackOverflow | ACC: 69.60 -> 73.16; 61.52 -> 65.21; 79.25 -> 83.73; 76.20 -> 83.07 | Helped |
| Add cluster-oriented contrastive learning (ablation) | GIFT-K · Twitter, MR, Snippets, StackOverflow | ACC: 71.49 -> 73.16; 63.24 -> 65.21; 82.89 -> 83.73; 81.42 -> 83.07 | Helped |
| Add SVD-based contrastive learning (ablation) | GIFT-SVD · Twitter, MR, Snippets, StackOverflow | ACC: 71.95 -> 73.16; 64.02 -> 65.21; 82.61 -> 83.73; 81.72 -> 83.07 | Helped |
| Use SVD instead of random TD-matrix masking to generate augmented views (ablation) | GIFT · Twitter, MR, Snippets, StackOverflow | ACC: 72.11 -> 73.16; 64.32 -> 65.21; 82.85 -> 83.73; 82.02 -> 83.07 | Helped |
| Use constrained seed k-means instead of classic k-means (ablation) | GIFT · Twitter, MR, Snippets, StackOverflow | ACC: 72.76 -> 73.16; 64.62 -> 65.21; 83.28 -> 83.73; 82.52 -> 83.07 | Helped |
| Vary the contrastive-learning temperature (ablation) | GIFT · StackOverflow | Test performance (ACC, F1) | Mixed |
| Increase the cluster-oriented contrastive loss coefficient ζ (ablation) | GIFT · StackOverflow | Test performance (ACC, F1) | Helped |
| Increase the SVD approximation rank r (ablation) | GIFT · StackOverflow | Test performance (ACC, F1) | Helped |
Surprising findings
- Fine-tuned BERT and RoBERTa perform poorly on short text with few labels, underperforming graph-based models on several datasets.
Limitations
- Relies on a fixed low rank r for SVD, with performance varying with rank in sensitivity analysis.
- Evaluated only on four short-text datasets with 40 labels per class setting.
Experimental setup
Datasets: Twitter MR Snippets StackOverflow
Benchmarks: Twitter MR Snippets StackOverflow
Metrics: accuracy macro F1
Baselines: TF-IDF+SVM LDA+SVM PTE CNN LSTM BERT RoBERTa TLGNN HyperGAT TextING DADGNN TextGCN
Builds on: SimCLR TextING NC-HGAT latent semantic indexing GCN
Closest papers by meaning
- A Simple Graph Contrastive Learning Framework for Short Text Classification (AAAI 2025 · similarity 0.94)
- Boosting Short Text Classification with Multi-Source Information Exploration and Dual-Level Contrastive Learning (AAAI 2025 · similarity 0.92)
- Contrastive Learning with Simplicial Convolutional Networks for Short-Text Classification (ICML 2025 · similarity 0.90)
- TGLsta: Low-resource Textual Graph Learning with Semantic and Topological Awareness via LLMs (AAAI 2025 · similarity 0.86)
- Leveraging Large Language Models for Node Generation in Few-Shot Learning on Text-Attributed Graphs (AAAI 2025 · similarity 0.85)
- Weakly-Supervised Contrastive Learning for Imprecise Class Labels (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.