Research map / Video / Action recognition

HyrCap: Hybrid Rank-Calibration of Action Proposals for Temporal Event Understanding

NeurIPS 2026 · Video › Action recognition · approach: Fully supervised temporal action detection

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HyrCap, a proposal-centric rank-calibration module that fuses InternVideo2 and V-JEPA2 proposals and recalibrates query-based TAD scores, and introduces a two-stage loc-first protocol for LLM-based semantic event understanding.

Problem

Query-based temporal action detection produces dense proposals whose ranking does not reliably reflect localization quality, cross-representation support, or inter-proposal redundancy, limiting proposal quality for downstream semantic understanding.

Approach

Treats dense query predictions as an over-complete hypothesis space and recalibrates proposal scores with a learned cross-representation consensus gate, a localization quality estimator, and a learned redundancy-suppression score, followed by temporal voting boundary refinement; also a two-stage localization-then-semantics evaluation protocol.

Key results

• On THUMOS14, HyrCap reaches 75.6 average mAP versus 73.8 for DiGIT with InternVideo2 features.
• On HACS-Segment, HyrCap achieves 47.1 average mAP versus 45.5 for DiGIT.
• On ActivityNet v1.3, HyrCap reaches 43.3 average mAP versus 42.0 for DiGIT.

Surprising findings

• V-JEPA2 is weaker than InternVideo2 as a detection representation yet yields fewer localization FPs in the top-5000 predictions, suggesting complementary localization evidence.

Limitations

• Depends on the quality of the base detector; missed events cannot be recovered by calibration.
• Auxiliary V-JEPA2 representation adds feature extraction, storage, and inference cost.
• Semantic evaluation is limited to predefined action categories and public TAD benchmarks, not open-vocabulary discovery.

Experimental setup

Datasets: THUMOS14 ActivityNet v1.3 HACS-Segment

Benchmarks: THUMOS14 ActivityNet v1.3 HACS-Segment

Metrics: mAP average mAP tIoU-based AP false positive breakdown

Baselines: DiGIT TE-TAD TadTR ActionFormer ActionMamba TriDet DyFADet AdaTAD ViT-TAD Direct Union + NMS Temporal WBF Fixed Consensus

Builds on: DiGIT TE-TAD DETR Soft-NMS IoU-Net

Closest papers by meaning

• Measure Twice, Cut Once: A Semantic-Oriented Approach to Video Temporal Localization with Video LLMs (ICLR 2026 · similarity 0.89)
• Decompose and Conquer: Compositional Reasoning for Zero-Shot Temporal Action Localization (AAAI 2026 · similarity 0.89)
• Temporal Context Aggregation Network for Temporal Action Proposal Refinement (CVPR 2021 · similarity 0.89)
• Fine-Grained Temporal Contrastive Learning for Weakly-Supervised Temporal Action Localization (CVPR 2022 · similarity 0.89)
• Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature (AAAI 2024 · similarity 0.88)
• Realigning Confidence with Temporal Saliency Information for Point-Level Weakly-Supervised Temporal Action Localization (CVPR 2024 · similarity 0.88)