Research map / Label-efficient and robust learning / Self-supervised and contrastive
Multi-aspect Self-guided Deep Information Bottleneck for Multi-modal Clustering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MSDIB, a deep information-bottleneck multi-modal clustering method that uses private, shared, and pseudo-label information as multi-aspect self-guidance, optimized via variational MI bounds.
Problem
Deep multi-modal clustering methods ignore guidance from both feature representations and cluster assignments and fail to remove redundant information between modalities.
Approach
Combines an information-compression term (minimizing MI between inputs and per-modality private features and among private features) with an information-preservation term (maximizing MI between private and shared features and between per-modality and global cluster assignments), trained end-to-end with a deep divergence-based clustering loss and a variational MI approximation.
Key results
- MSDIB reaches 67.8% ACC and 55.2% NMI on Caltech-2V, versus 64.1% and 52.9% for the best compared method DIVICE.
- On ESP-Game, MSDIB achieves 58.4% ACC and 37.5% NMI, a gain of about 4.6 ACC points over the best compared method.
- Ablation on the five datasets shows that adding both the information compression and preservation terms to the DDC loss improves ACC and NMI over DDC alone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MSDIB instead of MFLVC | multi-modal clustering · Caltech-2V | ACC: 61.5 -> 67.8 | Helped |
| Use MSDIB instead of MFLVC | multi-modal clustering · Caltech-2V | NMI: 53.6 -> 55.2 | Helped |
| Use MSDIB instead of MFLVC | multi-modal clustering · IAPR | ACC: 47.3 -> 50.6 | Helped |
| Use MSDIB instead of DIVICE | multi-modal clustering · IAPR | NMI: 23.0 -> 26.7 | Helped |
| Use MSDIB instead of MFLVC | multi-modal clustering · ESP-Game | ACC: 52.1 -> 66.1 | Helped |
| Use MSDIB instead of MFLVC | multi-modal clustering · ESP-Game | NMI: 39.4 -> 46.5 | Helped |
| Use MSDIB instead of MFLVC | multi-modal clustering · MIRFlickr | ACC: 53.8 -> 58.4 | Helped |
| Use MSDIB instead of DIVICE | multi-modal clustering · MIRFlickr | NMI: 33.5 -> 37.5 | Helped |
| Use MSDIB instead of MFLVC | multi-modal clustering · NUS-Wide | ACC: 45.1 -> 48.6 | Helped |
| Use MSDIB instead of MFLVC | multi-modal clustering · NUS-Wide | NMI: 33.1 -> 33.5 | Helped |
| Add information compression (LIC) (ablation) | DDC clustering · Caltech-2V, IAPR, ESP-Game, MIRFlickr, NUS-Wide | ACC and NMI: Caltech-2V: 63.4 -> 65.9 (ACC), 51.5 -> 52.9 (NMI); IAPR: 47.9 -> 47.7 (ACC), 24.8 -> 24.9 (NMI); ESP-Game: 59.5 -> 59.2 | Mixed |
| Add information preservation (LIP) (ablation) | DDC clustering · Caltech-2V, IAPR, ESP-Game, MIRFlickr, NUS-Wide | ACC and NMI: Caltech-2V: 63.4 -> 65.5 (ACC), 51.5 -> 54.1 (NMI); IAPR: 47.9 -> 50.2 (ACC), 24.8 -> 25.6 (NMI); ESP-Game: 59.5 -> 65.1 | Helped |
Surprising findings
- The method reaches 66.1% ACC on MIRFlickr-style ESP-Game comparison ...
- A free-parameter variant with two trade-off weights fixed to 1 performs nearly as well as the tuned MSDIB, suggesting low hyperparameter sensitivity.
Limitations
- Designed for complete and aligned data; may struggle with incomplete or unaligned multi-modal data.
- Trade-off parameters tuned by grid search over (0,1).
Experimental setup
Datasets: Caltech-2V ESP-Game IAPR MIRFlickr NUS-Wide
Benchmarks: Caltech-2V IAPR ESP-Game MIRFlickr NUS-Wide
Metrics: ACC NMI
Baselines: KM Ncuts AmKM AmNcuts CoregMVSC RMKMC SwMC ONMSC TBGL MMGC EAMC DEMC
Builds on: information bottleneck DCIB SIB-MSC deep divergence-based clustering (DDC)
Closest papers by meaning
- Super Deep Contrastive Information Bottleneck for Multi-modal Clustering (ICML 2025 · similarity 0.92)
- A Peer-review Look on Multi-modal Clustering: An Information Bottleneck Realization Method (ICML 2025 · similarity 0.89)
- Differentiable Information Bottleneck for Deterministic Multi-view Clustering (CVPR 2024 · similarity 0.87)
- Multi-Modal Proxy Learning Towards Personalized Visual Multiple Clustering (CVPR 2024 · similarity 0.85)
- Deep Multi-modal Graph Clustering via Graph Transformer Network (AAAI 2025 · similarity 0.84)
- Deep Variational Incomplete Multi-View Clustering: Exploring Shared Clustering Structures (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.