Research map / Graph learning / Graph self-supervised learning

Learning Subspace-Preserving Sparse Attention Graphs from Heterogeneous Multiview Data

NeurIPS 2026 · Graph learning › Graph self-supervised learning · approach: Multi-view graph clustering · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SAGL, which learns subspace-preserving sparse attention graphs via bilinear attention, a dynamic sparsity gate, and alpha-entmax (alpha=1.5) over heterogeneous multiview features from frozen pretrained backbones (DINOv3, SigLIP 2) for unsupervised clustering/transfer.

Problem

Unsupervised transfer learning from heterogeneous multiview features (from diverse pretrained models) fails to faithfully recover intrinsic subspace structure; dense or iterative sparse self-representation methods are costly and entangle subspaces.

Approach

Replace l0/l1 sparse self-representation with a differentiable alpha-entmax projection onto the probability simplex over asymmetric bilinear similarities, modulated by a learned feature-specific compression gate, so that sparse attention graphs are trained end-to-end without iterative solvers.

Key results

• On self-supervised clustering, SAGL reports ACC 97.57 on Pets and 79.34 on ImageNet-1K, best among compared methods including MSRL and TURTLE on all eight datasets.
• Ablation: removing sparse projection (SAGL_b, dense graphs) drops ImageNet-1K ACC to 18.86 and SUN397 ACC to 31.99 from 73.34.
• Against supervised linear probes, SAGL is higher on Pets, Flowers, EuroSAT, and Food101 but lower on KITTI, Caltech101, SUN397 and ImageNet-1K (e.g., KITTI 56.95 vs 76.60).

Surprising findings

• Adding more pretrained views does not consistently help; two-view combinations outperform some three-view combinations.
• Mixed-architecture pairs (e.g., ConvNeXt V2 with a ViT model) perform substantially worse than same-architecture ViT pairs.

Limitations

• Requires sufficient same-subspace samples in each mini-batch; shuffling may violate this and degrade the graph.
• Performance depends on view combination; heterogeneous architecture pairs yield poor graphs (per CKA analysis).
• Theoretical assumptions (independent subspaces, margin separation) are strong and only loosely connected to the practical method.

Experimental setup

Datasets: Pets KITTI Flowers Caltech101 EuroSAT SUN397 Food101 ImageNet-1K

Benchmarks: Pets KITTI Flowers Caltech101 EuroSAT SUN397 Food101 ImageNet-1K

Metrics: ACC NMI ARI classification accuracy

Baselines: SRLF PRR MIM TURTLE MSRL CLIP zero-shot transfer LaFTer supervised logistic regression probe

Builds on: alpha-entmax sparse subspace clustering MSRL TURTLE

Closest papers by meaning

• Fast Incomplete Multi-view Clustering with Adaptive Similarity Completion and Reconstruction (AAAI 2025 · similarity 0.86)
• Coeff-Tuning: A Graph Filter Subspace View for Tuning Attention-Based Large Models (CVPR 2025 · similarity 0.85)
• Deep Incomplete Multi-view Learning via Cyclic Permutation of VAEs (ICLR 2025 · similarity 0.85)
• Enhanced then Progressive Fusion with View Graph for Multi-View Clustering (CVPR 2025 · similarity 0.84)
• Sparse Video-Gen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity (ICML 2025 · similarity 0.84)
• Faster Video Diffusion with Trainable Sparse Attention (NeurIPS 2025 · similarity 0.84)