Research map / Probabilistic and scientific modeling / Variational and generative sampling
Differentiable Information Bottleneck for Deterministic Multi-view Clustering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Differentiable Information Bottleneck (DIB), a deterministic deep multi-view clustering method that estimates mutual information via normalized kernel Gram matrix eigenvalues (Rényi entropy) instead of variational bounds, combined with triplet consistency losses.
Problem
Deep multi-view clustering with information bottleneck relies on variational approximations of mutual information, which introduce estimation uncertainty in high-dimensional multi-view spaces.
Approach
Replaces variational MI lower bounds with a matrix-based Rényi α-order entropy computed from normalized Gaussian kernel Gram matrices, which is differentiable with analytical gradients, and uses it to train a deterministic view-specific encoder under an IB objective together with feature, cluster, and joint consistency terms.
Key results
- DIB reaches 99.86% ACC on MNIST-USPS and 99.00% ACC on BDGP, outperforming all 13 listed baselines in the reported table.
- On ESP, DIB obtains 59.06% ACC versus 58.45% for ConGMC, the strongest IB-based baseline shown in the table.
- Ablation on ESP shows ACC rising from 48.01% (clustering loss only) to 59.06% with the full loss including compression and joint consistency.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DIB instead of the strongest listed baseline | MVC methods · MNIST-USPS | ACC: 99.82 -> 99.86 | Helped |
| Use DIB instead of the strongest listed baseline | MVC methods · BDGP | ACC: 98.88 -> 99.00 | Helped |
| Use DIB instead of the strongest listed baseline | MVC methods · Handwritten | ACC: 88.80 -> 88.95 | Helped |
| Use DIB instead of the strongest listed baseline | MVC methods · ESP | ACC: 58.45 -> 59.06 | Helped |
| Use DIB instead of the strongest listed baseline | MVC methods · Flickr | ACC: 60.05 -> 60.40 | Helped |
| Use DIB instead of the strongest listed baseline | MVC methods · Caltech-3V | ACC: 73.37 -> 74.71 | Helped |
| Use DIB instead of the strongest listed baseline | MVC methods · Caltech-4V | ACC: 74.79 -> 75.64 | Helped |
| Use DIB instead of the strongest listed baseline | MVC methods · Caltech-5V | ACC: 83.84 -> 84.79 | Helped |
| Add feature consistency (Lfea) to cluster consistency (Lclu) (ablation) | DIB · MNIST-USPS, BDGP, and ESP | ACC: 85.72 -> 92.96; 82.58 -> 91.00; 48.01 -> 49.95 | Helped |
| Add joint consistency (Ljoint) to Lclu and Lfea (ablation) | DIB · MNIST-USPS, BDGP, and ESP | ACC: 92.96 -> 96.98; 91.00 -> 95.24; 49.95 -> 49.83 | Mixed |
| Add deterministic compression (Lcomp) to Lclu and Lfea (ablation) | DIB · MNIST-USPS, BDGP, and ESP | ACC: 92.96 -> 99.84; 91.00 -> 98.72; 49.95 -> 53.30 | Helped |
| Add joint consistency (Ljoint) to Lclu, Lfea, and Lcomp (ablation) | DIB · MNIST-USPS, BDGP, and ESP | ACC: 99.84 -> 99.86; 98.72 -> 99.00; 53.30 -> 59.06 | Helped |
Surprising findings
- The DCIB baseline scores 56.67% ACC on MNIST-USPS, far below its NMI of 72.38%, suggesting metric inconsistency in reported baselines.
Limitations
- Hyperparameters alpha and beta are fixed to 0.01 across datasets, with sensitivity only checked on two datasets.
- The Gram-matrix MI estimator's gradient derivation is brief, and the Rényi-entropy approximation to the true MI is not formally bounded in the text.
Experimental setup
Datasets: MNIST-USPS BDGP Handwritten ESP Flickr Caltech-3V Caltech-4V Caltech-5V
Benchmarks: MNIST-USPS BDGP Handwritten ESP Flickr Caltech-3V Caltech-4V Caltech-5V
Metrics: ACC NMI PUR
Baselines: BMVC SMKKM OPLFMVC FastMICE MFLVC CVCL AONGR GCFAgg SDMVC DMIM DMIB ConGMC
Builds on: information bottleneck contrastive clustering matrix-based Rényi entropy DMIB ConGMC
Closest papers by meaning
- Deep Variational Incomplete Multi-View Clustering: Exploring Shared Clustering Structures (AAAI 2024 · similarity 0.88)
- Investigating and Mitigating the Side Effects of Noisy Views for Self-Supervised Clustering Algorithms in Practical Multi-View Scenarios (CVPR 2024 · similarity 0.87)
- Robust Multi-view Clustering against Imperfect Information (NeurIPS 2026 · similarity 0.87)
- Mixture of Experts as Representation Learner for Deep Multi-View Clustering (AAAI 2025 · similarity 0.87)
- Multi-aspect Self-guided Deep Information Bottleneck for Multi-modal Clustering (AAAI 2025 · similarity 0.87)
- Towards Resource-friendly, Extensible and Stable Incomplete Multi-view Clustering (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.