Evaluating Confidence Calibration in Endoscopic Diagnosis Models

Nikoo Dehghani, Ayla Thijssen, Quirine E. W. Van Der Zander, Ramon-Michel Schreuder, Erik J. Schoon, Fons Van Der Sommen, Peter H. N. De With; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 5020-5025

Colorectal polyps are prevalent precursors to colorectal cancer making their accurate characterization essential for timely intervention and patient outcomes. Deep learning-based computer-aided diagnosis (CADx) systems have shown promising performance in the automated detection and categorization of colorectal polyps (CRP) using endoscopic images. However alongside the advancement in diagnostic accuracy the need for reliable and accurate quantification of uncertainty estimates within these systems has become increasingly important. The primary focus of this study is on refining the reliability of computer-aided diagnosis of CRPs within clinical practice. We perform an investigation of widely used model calibration techniques and how they translate into clinical applications specifically for CRP categorization data. The experiments reveal that the Variational Inference method excels in intra-dataset calibration but lacks efficiency and inter-dataset generalization. Laplace approximation and temperature scaling methods offer improved calibration across datasets.