Research map / Label-efficient and robust learning / Recognition and long-tail

CHASE: Learning Convex Hull Adaptive Shift for Skeleton-based Multi-Entity Action Recognition

NeurIPS 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Correspondence and causal representation learning · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CHASE, a sample-adaptive convex-hull-constrained origin shift wrapper plus a mini-batch pairwise MMD loss that reduces inter-entity distribution discrepancy and boosts single-entity skeleton backbones on multi-entity action recognition.

Problem

Single-entity skeleton action backbones perform poorly on multi-entity actions because entity skeleton distributions differ (non-i.i.d.), biasing backbone optimization when entities are encoded independently and averaged.

Approach

Adaptively reposition each multi-entity skeleton sequence's origin to a point inside its skeleton convex hull (via softmax-weighted combination of joints, with weights predicted by a lightweight Coefficient Learning Block), and add a Mini-batch Pair-wise MMD loss to minimize inter-entity discrepancies, acting as a plug-in normalization wrapper.

Key results

• On NTU Mutual 26 X-Sub, CTR-GCN improves from 89.32% to 91.30% with CHASE (+1.98).
• On H2O, CTR-GCN improves from 81.68% to 91.05% (+9.37) with CHASE.
• CHASE adds only about 1.8-2% trainable parameters (about 26.37k params on NTU Mutual 26 configuration) and about 2.50M FLOPs.

Measured effects reported in the paper

Surprising findings

• Removing the convex hull constraint drops accuracy by over 60% at lr 0.1 and 0.01, suggesting the constraint is critical for stable learning.
• Replacing the adaptive shift with a plain learned linear weighting (XWJ) causes a dramatic accuracy drop, showing trainable parameters alone are not sufficient.

Limitations

• Gains vary substantially across backbones and datasets; some categories slightly drop in accuracy.
• Improvements on ASB101 and group activity datasets are modest.
• Origin shift has no effect on Euclidean bone features unless k-hop bones are used.

Experimental setup

Datasets: NTU RGB+D NTU RGB+D 120 H2O Assembly101 Collective Activity Dataset Volleyball Dataset

Benchmarks: NTU Mutual 26 NTU Mutual 11 H2O Assembly101 Collective Activity Dataset Volleyball Dataset

Metrics: top-1 accuracy KLD JSD Bhattacharyya distance Hellinger distance MMD

Baselines: CTR-GCN InfoGCN STSA-Net HD-GCN ISTA-Net AHNet-Large SkeleTR me-GCN S2CoM BatchNorm ER

Builds on: CTR-GCN InfoGCN STSA-Net HD-GCN maximum mean discrepancy

Closest papers by meaning

• Recovering Complete Actions for Cross-dataset Skeleton Action Recognition (NeurIPS 2024 · similarity 0.86)
• Heterogeneous Skeleton-Based Action Representation Learning (CVPR 2025 · similarity 0.85)
• Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition (CVPR 2025 · similarity 0.85)
• Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition (CVPR 2025 · similarity 0.85)
• Skeleton-based Action Recognition with Non-linear Dependency Modeling and Hilbert-Schmidt Independence Criterion (AAAI 2025 · similarity 0.84)
• Semantic-guided Cross-Modal Prompt Learning for Skeleton-based Zero-shot Action Recognition (CVPR 2025 · similarity 0.84)