Eyes Tell the Truth: GazeVal Highlights Shortcomings of Generative AI in Medical Imaging

David Wong, Bin Wang, Gorkem Durak, Marouane Tliba, Akshay Chaudhari, Aladine Chetouani, Ahmet Enis Cetin, Cagdas Topel, Nicolo Gennaro, Camila Lopes Vendrami, Tugce Agirlar Trabzonlu, Amir Ali Rahsepar, Laetitia Perronne, Matthew Antalek, Onural Ozturk, Gokcan Okur, Andrew C Gordon, Ayis Pyrros, Frank H Miller, Amir Borhani, Hatice Savas, Eric Hart, Drew Torigian, Jayaram K udupa, Elizabeth Krupinski, Ulas Bagci; Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) Workshops, 2025, pp. 3167-3175

Abstract

The demand for high-quality synthetic data for model training and augmentation has never been greater in medical imaging. However, current evaluations predominantly rely on computational metrics that fail to align with human expert recognition. This leads to synthetic images that may appear realistic numerically but lack clinical authenticity, posing significant challenges in ensuring the reliability and effectiveness of AI-driven medical tools. To address this gap, we introduce GazeVal, a practical framework that synergizes expert eye-tracking data with direct radiological evaluations to assess the quality of synthetic medical images. GazeVal leverages gaze patterns of radiologists as they provide a deeper understanding of how experts perceive and interact with synthetic data in different tasks (i.e., diagnostic or Turing tests). Experiments with sixteen radiologists revealed that 96.6% of the generated images (by the most recent state-of-the-art AI algorithm) were identified as fake, demonstrating the limitations of generative AI in producing clinically accurate images.

Related Material

[pdf] [arXiv]
[bibtex]
@InProceedings{Wong_2025_CVPR, author = {Wong, David and Wang, Bin and Durak, Gorkem and Tliba, Marouane and Chaudhari, Akshay and Chetouani, Aladine and Cetin, Ahmet Enis and Topel, Cagdas and Gennaro, Nicolo and Vendrami, Camila Lopes and Trabzonlu, Tugce Agirlar and Rahsepar, Amir Ali and Perronne, Laetitia and Antalek, Matthew and Ozturk, Onural and Okur, Gokcan and Gordon, Andrew C and Pyrros, Ayis and Miller, Frank H and Borhani, Amir and Savas, Hatice and Hart, Eric and Torigian, Drew and udupa, Jayaram K and Krupinski, Elizabeth and Bagci, Ulas}, title = {Eyes Tell the Truth: GazeVal Highlights Shortcomings of Generative AI in Medical Imaging}, booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) Workshops}, month = {June}, year = {2025}, pages = {3167-3175} }