Vision-Based Treatment Localization with Limited Data: Automated Documentation of Military Emergency Medical Procedures

Trevor Powers, Elaheh Hatamimajoumerd, William Chu, Vishakk Rajendran, Rishi Shah, Frank Diabour, Marc Vaillant, Richard Fletcher, Sarah Ostadabbas; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 1819-1828

In response to the challenges faced in documenting medical procedures in military settings, where time constraints and cognitive load limit the completion of life-saving Tactical Combat Casualty Care (TCCC) Cards, we present a novel end-to-end computer vision pipeline for autonomous detection and documentation of common military emergency medical treatments. Our pipeline is specifically designed to handle limited and challenging data encountered in military scenarios. To support the development of this pipeline, we introduce SimTrI, a labeled dataset comprising 116 twenty-second videos capturing patients undergoing four prevalent treatment procedures. Our pipeline incorporates training and fine-tuning of object detection and human pose estimation models, complemented by a proprietary pose-enhancement algorithm and a range of unique filtering and post-processing techniques. Through comprehensive development and optimization, our pipeline achieves exceptional performance, demonstrating 100% precision and 62% recall on our dedicated 23-video test set. Furthermore, the pipeline automates the generation of TCCC-relevant information, significantly improving the efficiency of TCCC documentation. Comparative analysis against previous state-of-the-art techniques in emergency medical autonomous documentation demonstrates that our pipelineperforms exceptionally