GazeSearch: Radiology Findings Search Benchmark

Trong Thang Pham, Tien-Phat Nguyen, Yuki Ikebe, Akash Awasthi, Zhigang Deng, Carol C. Wu, Hien Nguyen, Ngan Le; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 96-106

Medical eye-tracking data is an important information source for understanding how radiologists visually interpret medical images. This information not only improves the accuracy of deep learning models for X-ray analysis but also their interpretability enhancing transparency in decision-making. However the current eye-tracking data is dispersed unprocessed and ambiguous making it difficult to derive meaningful insights. Therefore there is a need to create a new dataset with more focus and purposeful eyetracking data improving its utility for diagnostic applications. In this work we propose a refinement method inspired by the target-present visual search challenge: there is a specific finding and fixations are guided to locate it. After refining the existing eye-tracking datasets we transform them into a curated visual search dataset called GazeSearch specifically for radiology findings where each fixation sequence is purposefully aligned to the task of locating a particular finding. Subsequently we introduce a scan path prediction baseline called ChestSearch specifically tailored to GazeSearch. Finally we employ the newly introduced GazeSearch as a benchmark to evaluate the performance of current state-of-the-art methods offering a comprehensive assessment for visual search in the medical imaging domain. Code is available at https://github.com/ UARK-AICV/GazeSearch.