Research map / Vision-language and multimodal / Cross-modal learning

Unpaired Canonical Correlation Analysis

NeurIPS 2026 · Vision-language and multimodal › Cross-modal learning · approach: Multi-modality image fusion

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Unpaired CCA (UCCA), which learns maximally correlated linear projections from strictly unpaired multiview data by linking the maximum-correlation pairing to a linear-kernel QAP solved on K-Means anchors.

Problem

Canonical Correlation Analysis requires paired samples across views, which are often unavailable; the paper asks how to recover maximally correlated components when no correspondences are given during training.

Approach

Defines a proxy pairing (MCP over orthogonal projections), proves MCP over the scaled Frobenius sphere equals a linear-kernel Koopmans-Beckmann QAP under a majorization assumption, then matches K-Means anchors via approximate QAP solvers and runs CCA on the pseudo-pairs.

Key results

• UCCA reaches total correlation 1.239 on PC-KL and 1.957 on KL-PA, versus 0.301 or lower for the best unpaired baseline in those configurations, approaching paired CCA (1.979, 1.990).
• On SNARE cross-view kNN classification UCCA achieves 0.849 accuracy versus 0.650 for SCOTv1-CCA, against paired CCA at 0.887.
• Empirical Kendall Tau distances between QAP-based and projected MCP pairings are heavily concentrated at zero, supporting the MCP_O(d)=MCP_S_F equivalence in practice.

Surprising findings

• Naive maximum correlation pairing without projections fails under arbitrary rotations of the views, motivating the orthogonal-projection-aware proxy.

Limitations

• Performance gap remains relative to paired CCA upper bound.
• Framework limited to bi-view setting.
• Equivalence theorem relies on Assumption (weak majorization), only validated empirically for real data.

Experimental setup

Datasets: SNARE Flickr COCO Handwritten digits benchmark

Benchmarks: Handwritten (PC-KL, PC-PA, KL-PA) SNARE Flickr COCO

Metrics: total correlation kNN cross-view classification accuracy Kendall Tau distance

Baselines: J-MDS-CCA SCOTv1-CCA SCOTv2-CCA SCA UCA

Builds on: Canonical Correlation Analysis Koopmans-Beckmann QAP SCA UCA SCOT

Closest papers by meaning

• Unsupervised Correlation Analysis (CVPR 2018 · similarity 0.89)
• Nonparametric Canonical Correlation Analysis (ICML 2016 · similarity 0.89)
• Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning (ICLR 2024 · similarity 0.89)
• On Sparse Canonical Correlation Analysis (NeurIPS 2024 · similarity 0.87)
• Learning Shared Representations from Unpaired Data (NeurIPS 2025 · similarity 0.85)
• TreeVI: Reparameterizable Tree-structured Variational Inference for Instance-level Correlation Capturing (NeurIPS 2024 · similarity 0.83)