Research map / Label-efficient and robust learning / Self-supervised and contrastive

High-Order Contrastive Learning with Fine-grained Comparative Levels for Sparse Ordinal Tensor Completion

ICML 2024 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Inductive bias and minimal representations

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HOCTC, a contrastive tensor-completion model with query-expanded self-attention (SAQE) and multi-level contrastive sampling over ordinal tensor entries, evaluated on spatio-temporal and recommender sparse tensors.

Problem

Extending contrastive learning to high-order interactions and fine-grained ordinal comparison levels for sparse ordinal tensor completion, where self-attention has only as many queries as tensor modes and negative samples grow combinatorially.

Approach

Expands the few attention queries of a tensor triple via learned linear combinations of raw queries (SAQE), and replaces binary positive/negative contrast with three levels (positive, weakly-positive, negative) where weakly-positives are found efficiently by mode-index sorting of observed entries.

Key results

• On PEMS, HOCTC reaches MAE 3.4033 vs. best baseline LightNestle 4.0680 (about 16.3% relative MAE improvement); it attains the lowest error in 11 of 12 spatio-temporal comparisons and 8 of 12 recommender comparisons.
• On GOWALLA, HOCTC achieves MAE 0.4652 and MAPE 16.1613, about 26.7% and 57.9% relative improvements over the best baseline in MAE and MAPE.
• Ablation on MAE: removing both SAQE and MLCS gives the worst results; using both gives the lowest MAE on all 8 tensors (e.g., PEMS 3.8511 to 3.4033).

Measured effects reported in the paper

Surprising findings

• HOCTC's RMSE on GOWALLA (1.1920) is higher than several baselines (e.g., NTC 1.1545), so it does not win on all metrics.
• Increasing SAQE queries can cause slight performance fluctuation on very sparse tensors such as Gowalla, attributed to possible overfitting.

Limitations

• Binary tensors such as knowledge graphs are not studied and are deferred to future work.
• Spatio-temporal experiments use only weakly-positive samples because those tensors are complete before sampling, so the negative-sample component is not exercised there.
• RMSE is not uniformly best across datasets (e.g., GOWALLA), and the authors note a trade-off among error metrics.

Experimental setup

Datasets: PEMS GZSPEED METR CITYTEMP SG BEAUTY MOVIE&TV GOWALLA

Benchmarks: PEMS GZSPEED METR CITYTEMP SG BEAUTY MOVIE&TV GOWALLA

Metrics: MAE MAPE RMSE

Baselines: P-Tucker NeuralCP CoSTCo NTM NTC LightNestle

Builds on: Transformer self-attention AutoInt CoSTCo contrastive learning (InfoNCE-style loss)

Closest papers by meaning

• Coarse-To-Fine Tensor Trains for Compact Visual Representations (ICML 2024 · similarity 0.85)
• Self-Attention Driven Tensor Representation for High-Order Data Recovery (CVPR 2026 · similarity 0.85)
• Contrastive Order Learning: A General Framework for Ordinal Regression (ICML 2026 · similarity 0.85)
• Tensor denoising and completion based on ordinal observations (ICML 2020 · similarity 0.85)
• Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods (NeurIPS 2024 · similarity 0.83)
• Contrastive Multi-view Subspace Clustering via Tensor Transformers Autoencoder (AAAI 2025 · similarity 0.83)