Research map / Vision-language and multimodal / Cross-modal learning

RePercENT: Scaling Disentangled Representation Learning Beyond Two Modalities

NeurIPS 2026 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal fusion with missing modalities · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RePercENT, a Perceiver-based self-supervised framework that jointly learns pairwise unique and shared disentangled representations over pre-extracted embeddings, scaling linearly in the number of modalities, with theoretical optimality guarantees.

Problem

Multimodal disentanglement of unique and shared information is confined to two modalities due to exponential growth of atomic components and lack of tractable objectives beyond two modalities.

Approach

Model pairwise unique and shared components per modality pair, use one Perceiver-style disentangling module per modality with semantic pair/type encodings and group slot attention, and optimize all pairwise components jointly with InfoNCE, KL, and cross-covariance surrogate losses.

Key results

• On IRFL, RePercENT with semantic encoding and group slot attention reaches 44.07% overall top-1 accuracy, versus 41.73% for the best CLIP variant and 40.30% for the ablation without both mechanisms.
• On TCGA molecular data, disentangled decompositions improve over HONeYBEE embeddings by up to +59.4 points (KIRC) and show large gains on several cohorts, though with some drops (e.g., -2.6 on LUAD).
• RePercENT is reported to be the only evaluated method with linear scaling in the number of modalities, and gMLP needs about 2x parameters and 7x FLOPs without improving IRFL performance.

Surprising findings

• The full decomposition D_ij outperforms the unique and shared components individually on most cancer types, which the authors argue challenges the multi-view redundancy assumption.
• Only RePercENT and gMLP recover all components in the synthetic setting, while GRU and MLP suffer representation collapse as the number of modalities grows.

Limitations

• Focuses only on pairwise components and does not model higher-order interactions among three or more modalities.
• Performance depends on the quality of frozen pre-extracted foundation model embeddings.
• Evaluation on IRFL and TCGA uses a limited set of modalities and tasks; the synthetic setting is controlled.

Experimental setup

Datasets: IRFL TCGA (via HONeYBEE embeddings) synthetic Gaussian data

Benchmarks: IRFL TCGA cancer type classification synthetic disentanglement linear probing

Metrics: top-1 accuracy macro-F1 Delta model deviation from ideal disentanglement parameter count FLOPs

Baselines: CLIP zero-shot CLIP projection head CLIP fine-tuning gMLP GRU MLP

Builds on: Perceiver Slot Attention InfoNCE Minimum Necessary Information FactorCL

Closest papers by meaning

• An Information Criterion for Controlled Disentanglement of Multimodal Data (ICLR 2025 · similarity 0.92)
• Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Models (NeurIPS 2024 · similarity 0.88)
• Unity by Diversity: Improved Representation Learning for Multimodal VAEs (NeurIPS 2024 · similarity 0.87)
• Disentanglement Beyond Static vs. Dynamic: A Benchmark and Evaluation Framework for Multi-Factor Sequential Representations (NeurIPS 2025 · similarity 0.87)
• Disentangled Representation Learning via Modular Compositional Bias (NeurIPS 2025 · similarity 0.87)
• Transferring disentangled representations: bridging the gap between synthetic and real images (NeurIPS 2024 · similarity 0.86)