Research map / Vision-language and multimodal / Cross-modal learning

A Peer-review Look on Multi-modal Clustering: An Information Bottleneck Realization Method

ICML 2025 · Vision-language and multimodal › Cross-modal learning · approach: Medical multimodal fusion

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Peer-review Trustworthy Information Bottleneck (PTIB), a parameter-free weighted multi-modal clustering method that scores each modality as a peer-review "author" and adds a self-supervised trustworthiness score for modality weights.

Problem

Weighted multi-modal clustering methods learn modality weights that are untrustworthy, learned in isolation from cross-modal relations, and depend on extra hyperparameters.

Approach

Treat each modality alternately as an "author" and the others as "reviewers": reviewer scores are normalized mutual information between local clusterings, weighted by a trustworthiness score computed from cluster distortion against the final clustering (self-supervised), and the resulting weights drive an information-bottleneck clustering objective.

Key results

• PTIB reaches 99.80% Acc on 20NG and 93.33% Acc on COIL20, outperforming the best second-place method by 0.40 and 6.11 points respectively.
• Across eight datasets, PTIB improves Acc over the second-best method by 0.4 to 6.79 points.
• The parameter-free version (beta = infinity) is typically about 2 points lower in Acc than tuned PTIB, with the largest drop of 6.87 points on COIL20.

Measured effects reported in the paper

Surprising findings

• Concatenating all modalities (Ncuts-All) can substantially underperform the best single-modal method, e.g. 28.55% lower Acc on COIL20.

Limitations

• Designed only for fully aligned and complete multi-modal data; unaligned, missing, or corrupted samples are not handled.
• Requires the number of clusters to be specified in advance.
• Two-modality case needs a special prejudice-handling rule because the peer-review score is symmetric and carries no information.

Experimental setup

Datasets: 20NG COIL20 Event Soccer 17Flowers 75Flowers COIL100 MMI

Benchmarks: 20NG COIL20 Event Soccer 17Flowers 75Flowers COIL100 MMI

Metrics: Accuracy NMI

Baselines: KM Ncuts MVIB Co(reg) MfIB RMSC LMSC MLAN GMC DMIB FPMVS-CAG MCMLE

Builds on: information bottleneck normalized mutual information draw-and-merge clustering

Closest papers by meaning

• Multi-aspect Self-guided Deep Information Bottleneck for Multi-modal Clustering (AAAI 2025 · similarity 0.89)
• Super Deep Contrastive Information Bottleneck for Multi-modal Clustering (ICML 2025 · similarity 0.88)
• Multi-View Clustering by Inter-cluster Connectivity Guided Reward (ICML 2024 · similarity 0.84)
• Differentiable Information Bottleneck for Deterministic Multi-view Clustering (CVPR 2024 · similarity 0.84)
• Medusa: A Multi-Scale High-order Contrastive Dual-Diffusion Approach for Multi-View Clustering (CVPR 2025 · similarity 0.84)
• A Non-parametric Graph Clustering Framework for Multi-View Data (AAAI 2024 · similarity 0.84)