Research map / Label-efficient and robust learning / Recognition and long-tail
View-Category Interactive Sharing Transformer for Incomplete Multi-View Multi-Label Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VIST, a two-layer view-category interactive transformer with KNN-style Gaussian missing-view generation and contrastive view-category consistency loss, for doubly incomplete multi-view multi-label classification.
Problem
Multi-view multi-label classification when both views and labels are missing (double incompleteness), where existing methods ignore view-category complementarity and do not impute missing views.
Approach
A transformer that alternates interaction between view tokens and category tokens (with shared view/category feature propagation), a KNN-guided multivariate Gaussian imputation of missing view embeddings using category vectors, and a contrastive/correlation-alignment loss guided by label co-occurrence to enhance embeddings.
Key results
- On Corel5k AP, VIST reaches 41.5 vs 38.1 for the second-best LMVCAT under 50% missing views and labels.
- With full views and labels, VIST reaches 55.4 AP on Corel5k vs 52.1 for LMVCAT.
- Ablation on Corel5k: the transformer alone gives 39.5 AP, adding consistency loss 40.4, and adding missing view generation 41.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the VIST method instead of the strongest baseline | VIST · Corel5k | AP: 38.4±0.4 -> 41.5±0.2 | Helped |
| Use the VIST method instead of the strongest baseline | VIST · Pascal07 | AP: 53.6±0.2 -> 53.9±0.3 | Helped |
| Use the VIST method instead of the strongest baseline | VIST · Espgame | AP: 29.7±0.2 -> 30.7±0.3 | Helped |
| Use the VIST method instead of the strongest baseline | VIST · IAPRTC12 | AP: 32.3±0.1 -> 33.9±0.2 | Helped |
| Use the VIST method instead of the strongest baseline | VIST · Mirflickr | AP: 59.8±0.2 -> 60.4±0.3 | Helped |
| Replace four vanilla transformer encoders with the view-category interactive sharing transformer (ablation) | VIST · Corel5k | AP: 34.9 -> 39.5 | Helped |
| Add view-category consistency guided embedding enhancement (ablation) | VIST · Corel5k | AP: 39.5 -> 40.4 | Helped |
| Add missing view generation (ablation) | VIST · Corel5k | AP: 40.4 -> 41.5 | Helped |
| Add masked asymmetric loss La (ablation) | VIST · Corel5k | AP: 40.6 -> 41.5 | Helped |
| Add masked asymmetric loss La (ablation) | VIST · Corel5k | AUC: 89.8 -> 90.2 | Helped |
Surprising findings
- Missing labels hurt performance more than missing views, per the missing-ratio analysis.
Limitations
- Performance degrades as the view missing ratio grows, since imputation relies on existing views and degrades toward random generation.
- Baselines had to be modified to handle incomplete inputs, which may affect fairness of comparison.
- Evaluated only on image datasets with six handcrafted feature views.
Experimental setup
Datasets: Corel5k Pascal07 Espgame IAPRTC12 Mirflickr
Benchmarks: Corel5k Pascal07 Espgame IAPRTC12 Mirflickr
Metrics: AP 1-RL AUC
Baselines: lrMMC MVL-IV iMSF GLOCAL iMvWL NAIML DDINet DICNet LMVCAT
Builds on: Transformer encoder SimCLR-style contrastive loss asymmetric loss
Closest papers by meaning
- Incomplete Multi-View Multi-label Learning via Disentangled Representation and Label Semantic Embedding (CVPR 2025 · similarity 0.91)
- Learning Compact Semantic Information for Incomplete Multi-View Missing Multi-Label Classification (ICML 2025 · similarity 0.91)
- Attention-Induced Embedding Imputation for Incomplete Multi-View Partial Multi-Label Classification (AAAI 2024 · similarity 0.90)
- Semi-Supervised Multi-View Multi-Label Learning with View-Specific Transformer and Enhanced Pseudo-Label (AAAI 2025 · similarity 0.90)
- A Two-Stage Information Extraction Network for Incomplete Multi-View Multi-Label Classification (AAAI 2024 · similarity 0.88)
- Partial Multi-View Multi-Label Classification via Semantic Invariance Learning and Prototype Modeling (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.