Research map / Label-efficient and robust learning / Semi-supervised learning
STiL: Semi-supervised Tabular-Image Learning for Comprehensive Task-Relevant Information Exploration in Multimodal Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STiL, a semi-supervised image-tabular learning framework that combines disentangled contrastive consistency, consensus-guided pseudo-labeling, and prototype-guided label smoothing to recover task-relevant modality-specific and shared information from few labels.
Problem
Multimodal image-tabular classification with limited labeled data; existing SSL and multimodal SemiSL methods focus on shared or unimodal features and miss task-relevant modality-specific information (the "Modality Information Gap").
Approach
Jointly disentangles modality-shared and modality-specific features with contrastive consistency and CLUB-based mutual-information minimization, generates pseudo-labels via rule-based consensus among a multimodal and two unimodal classifiers, and smooths pseudo-labels with prototype similarity stored as class-level embeddings.
Key results
- On 1% labeled DVM, STiL reaches 91.92% accuracy vs 90.45% for the best SemiSL baseline (Self-KD) and 88.93% for the best SSL baseline (TIP linear probing).
- On 1% labeled Infarction, STiL achieves 82.64% AUC, 3.29 points above the best SemiSL baseline and 7.01 points above the best SSL baseline.
- Ablation on 1% DVM shows each of DCC, CGPL and PGLS improves over the supervised baseline (82.98%), with the full model at 91.92%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use STiL instead of TIP | multimodal image-tabular classification · DVM, 1% labeled data | accuracy (%): 88.93 -> 91.92 | Helped |
| use STiL instead of SAINT | multimodal image-tabular classification · CAD, 1% labeled data | AUC (%): 78.02 -> 83.54 | Helped |
| use STiL instead of SAINT | multimodal image-tabular classification · Infarction, 1% labeled data | AUC (%): 75.63 -> 82.64 | Helped |
| use STiL instead of Self-KD | multimodal image-tabular classification · DVM, 1% labeled data | accuracy (%): 90.45 -> 91.92 | Helped |
| use STiL instead of Co-training | multimodal image-tabular classification · CAD, 1% labeled data | AUC (%): 81.00 -> 83.54 | Helped |
| add DCC, CGPL, and PGLS to the labeled-only baseline (ablation) | STiL · DVM, CAD, and Infarction, 1% labeled data | accuracy (%) / AUC (%): 82.98 -> 91.92; 75.75 -> 83.54; 73.70 -> 82.64 | Helped |
| add DCC to the baseline (ablation) | STiL · DVM, CAD, and Infarction, 1% labeled data | accuracy (%) / AUC (%): 82.98 -> 89.43; 75.75 -> 81.26; 73.70 -> 74.04 | Mixed |
| add CGPL to the baseline (ablation) | STiL · DVM, CAD, and Infarction, 1% labeled data | accuracy (%) / AUC (%): 82.98 -> 83.82; 75.75 -> 83.59; 73.70 -> 78.39 | Helped |
| add contrastive consistency without disentanglement losses (ablation) | STiL baseline · CAD and Infarction, 1% labeled data | AUC (%): 75.75 -> 80.95; 73.70 -> 71.70 | Mixed |
| add disentanglement losses to contrastive consistency (ablation) | STiL baseline · CAD and Infarction, 1% labeled data | AUC (%): 80.95 -> 83.59; 71.70 -> 78.39 | Helped |
| remove intra- and inter-modality interaction (ablation) | STiL · DVM, CAD, and Infarction, 1% labeled data | accuracy (%) / AUC (%): 90.83 vs 91.92; 81.25 vs 83.54; 81.03 vs 82.64 | Hurt |
| remove consensus collaboration from pseudo-labeling (ablation) | STiL · DVM, CAD, and Infarction, 1% labeled data | accuracy (%) / AUC (%): 89.88 vs 91.92; 82.23 vs 83.54; 75.08 vs 82.64 | Hurt |
Surprising findings
- Adding contrastive consistency alone to the baseline lowers Infarction AUC (73.70 to 71.70), while the disentangled version recovers and improves it.
- Combining DCC and CGPL without PGLS performs worse than either alone on the CAD task.
Limitations
- Evaluated only on image-tabular data; generalization to other modalities such as text and video is left to future work.
- Disentanglement loss can interfere with pseudo-label generation on some tasks (CAD results when DCC and CGPL are combined without PGLS).
Experimental setup
Datasets: DVM UK Biobank
Benchmarks: DVM UK Biobank CAD UK Biobank Infarction
Metrics: accuracy AUC
Baselines: ResNet-50 DAFT Interact Fuse TIP SimCLR BYOL SCARF SAINT MMCL CoMatch SimMatch FreeMatch
Builds on: FixMatch-style weak-strong consistency TIP MMCL CLUB SimMatch
Closest papers by meaning
- Semi-Supervised Multimodal Classification Through Learning from Modal and Strategic Complementarities (AAAI 2025 · similarity 0.89)
- Seek Common Ground While Reserving Differences: Semi-Supervised Image-Text Sentiment Recognition (CVPR 2025 · similarity 0.86)
- InterLUDE: Interactions between Labeled and Unlabeled Data to Enhance Semi-Supervised Learning (ICML 2024 · similarity 0.86)
- MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image (NeurIPS 2026 · similarity 0.85)
- Facilitating Multimodal Classification via Dynamically Learning Modality Gap (NeurIPS 2024 · similarity 0.85)
- MIDAS: Misalignment-based Data Augmentation Strategy for Imbalanced Multimodal Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.