Psych-Occlusion: using Visual Psychophysics for Aerial Detection of Occluded Persons during Search and Rescue

Arturo Miguel Russell Bernal, Jane Cleland-Huang, Walter Scheirer; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 3383-3395

The success of Emergency Response (ER) scenarios such as search and rescue is often dependent upon the prompt location of a lost or injured person. With the increasing use of small Unmanned Aerial Systems (sUAS) as "eyes in the sky" during ER scenarios efficient detection of persons from aerial views plays a crucial role in achieving a successful mission outcome. Fatigue of human operators during prolonged ER missions coupled with limited human resources highlights the need for sUAS equipped with Computer Vision (CV) capabilities to aid in finding the person from aerial views. However the performance of CV models onboard sUAS substantially degrades under real-life rigorous conditions of a typical ER scenario where person search is hampered by occlusion and low target resolution. To address these challenges we extracted images from the NOMAD dataset and performed a crowdsource experiment to collect behavioural measurements when humans were asked to "find the person in the picture". We exemplify the use of our behavioral dataset Psych-ER by using its human accuracy data to adapt the loss function of a detection model. We tested our loss adaptation on a RetinaNet model evaluated on NOMAD against increasing distance and occlusion with our psychophysical loss adaptation showing improvements over the baseline at higher distances across different levels of occlusion without degrading performance at closer distances. To the best of our knowledge our work is the first human-guided approach to address the location task of a detection model while addressing real-world challenges of aerial search and rescue. All datasets and code can be found at: https://github.com/ArtRuss/NOMAD.