Research map / Video / Action recognition
Polyphony: Diffusion-based Dual-Hand Action Segmentation with Alternating Vision Transformer and Semantic Conditioning
Hao Zheng, Hu Wang, Tiantian Zheng, Prajjwal Bhattarai, Tuka Alhanai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-hand feature fusion (ablation) | Polyphony · HA-ViD LH | accuracy: 55.5 -> 57.1 | Improved |
| add cross-hand feature fusion (ablation) | Polyphony · HA-ViD RH | accuracy: 59.5 -> 60.6 | Improved |
| add adaptive loss weighting on top of feature fusion (ablation) | Polyphony · HA-ViD LH | accuracy: 55.3 -> 55.5 | Improved |
| add action (classification logit) features to motion features (ablation) | Polyphony · HA-ViD LH | accuracy: 56.0 -> 55.8 | No clear effect |
| add action (classification logit) features to motion features (ablation) | Polyphony · HA-ViD RH | accuracy: 58.0 -> 58.3 | No clear effect |
| add semantic features (ablation) | Polyphony · HA-ViD RH | accuracy: 58.3 -> 60.6 | Improved |
| add semantic features (ablation) | Polyphony · HA-ViD LH | accuracy: 55.8 -> 57.1 | Improved |
| use alternating instead of joint training (ablation) | Polyphony ADH-ViT · HA-ViD side-view segmentation LH | accuracy: 53.0 -> 56.9 | Improved |
| use alternating instead of joint training (ablation) | Polyphony ADH-ViT · HA-ViD side-view segmentation RH | accuracy: 55.4 -> 55.7 | Improved |
| add random clip sampling to segment-based sampling (ablation) | ADH-ViT · HA-ViD action recognition LH | top-1 accuracy: 47.0 -> 60.1 | Improved |
| use structured compositional descriptions instead of naive (ablation) | MiniLM-L6 semantic conditioning · HA-ViD side-view | accuracy: 55.0 -> 57.4 | Improved |
| use larger language model for semantic embedding (BGE-large vs MiniLM-L6) (ablation) | Polyphony · HA-ViD side-view | accuracy: 57.4 -> 56.6 | Hurt |
Closest papers by meaning
- GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities (CVPR 2025 · similarity 0.87)
- BOTH2Hands: Inferring 3D Hands from Both Text Prompts and Body Dynamics (CVPR 2024 · similarity 0.86)
- FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation (CVPR 2025 · similarity 0.86)
- The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction (NeurIPS 2026 · similarity 0.86)
- StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Video (NeurIPS 2026 · similarity 0.86)
- SiMHand: Mining Similar Hands for Large-Scale 3D Hand Pose Pre-training (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
