Research map / Detection and segmentation / Tracking
CholecTrack20: A Multi-Perspective Tracking Dataset for Surgical Tools
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CholecTrack20, a laparoscopic cholecystectomy dataset of 20 full-length videos annotated at 1 fps with multi-perspective (intraoperative, intracorporeal, visibility) tool tracks, and benchmarks detectors and MOT trackers on it.
Problem
Existing surgical tool tracking datasets use generic tracking formalizations that miss tools leaving the camera view or the body, and lack context-rich annotations, limiting clinically relevant tool trajectories.
Approach
Redefines tool tracking with three perspectives (intraoperative, intracorporeal, visibility) and provides fine-grained labels for all three on the same laparoscopic videos, along with operator, phase, and visual challenge annotations.
Key results
- YOLOv7 achieves the highest detection AP of 56.1% (AP0.5:0.95) and 80.6% AP0.5 among evaluated detectors.
- Bot-SORT attains the best tracking HOTA of 44.7% on visibility trajectories, 27.0% on intracorporeal and 17.4% on intraoperative trajectories.
- Intracorporeal tracking is the most challenging perspective, with a maximum HOTA of 27.0%, and the authors conclude current methods fall short of clinical translation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bot-SORT instead of ByteTrack for visibility tracking | Multi-object tracking on CholecTrack20 · Visibility trajectory | HOTA: 41.5 -> 44.7 | Helped |
| Track from the visibility perspective rather than intraoperatively | Bot-SORT · CholecTrack20 | HOTA: 17.4 -> 44.7 | Helped |
| Track from the visibility perspective rather than intracorporeally | Bot-SORT · CholecTrack20 | HOTA: 27.0 -> 44.7 | Helped |
| Use YOLOv9 instead of YOLOv7 for tool detection | CholecTrack20 tool detection · CholecTrack20 | AP0.5:0.95: 56.1 -> 56.5 | Helped |
Surprising findings
- Intracorporeal tracking is harder than intraoperative tracking, with the best HOTA only 27.0% versus 17.4% for intraoperative (the intraoperative figure is lower in the text but Bot-SORT's intraoperative score is reported as 17.4%).
Limitations
- Current SOTA trackers achieve under 45% HOTA, insufficient for clinical translation.
- Intracorporeal and out-of-view tool status are hard to infer from visual cues alone, limiting re-identification.
- Annotations cover only laparoscopic cholecystectomy from a single hospital source.
Experimental setup
Datasets: Cholec80 CholecT50 CholecTrack20
Benchmarks: CholecTrack20
Metrics: AP AP0.5 AP0.75 HOTA DetA AssA LocA MOTA MOTP IDF1
Baselines: OC-SORT FairMOT TransTrack ByteTrack Bot-SORT SMILETrack SORT Faster-RCNN Cascade-RCNN CenterNet FCOS SSD
Builds on: Cholec80 CholecT50 HOTA ByteTrack
Closest papers by meaning
- Bridging Vision and Language for Robust Context-Aware Surgical Point Tracking: The VL-SurgPT Dataset and Benchmark (AAAI 2026 · similarity 0.89)
- EgoExOR: An Ego-Exo-Centric Operating Room Dataset for Surgical Activity Understanding (NeurIPS 2025 · similarity 0.86)
- SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmark (CVPR 2026 · similarity 0.85)
- MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environments (CVPR 2025 · similarity 0.85)
- SurgVista: Long-Horizon Surgical World Modeling with Plausible Instrument-Tissue Dynamics (NeurIPS 2026 · similarity 0.84)
- Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Data (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.