On the Risk of Manual Annotations in 3D Confocal Microscopy Image Segmentation

Justin Sonneck, Shuo Zhao, Jianxu Chen; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 3894-3902

Image segmentation in 3D confocal fluorescence microscopy images is a common problem in many biomedical studies. Deep learning-based methods have achieved great success on such tasks. In the literature, manual 3D annotations are still commonly used for model training or performance evaluation. But, due to the nature of the lens-based optical instruments, diffraction of light always occurs, which can lead to obscure boundaries of the biomedical structures being imaged. For example, when analyzing nuclei from 3D fluorescence microscopy images of cells marked by DNA dyes, the exact boundaries are usually not clearly identifiable, especially along Z. This makes accurate segmentation, both manually and automatically, very challenging. For applications where the boundary accuracy is crucial, the downstream analyses can thus be significantly compromised. This problem can be addressed with special experimental-computational co-design to acquire the "biological ground truth". For the nuclei example, we can take cells expressing mEGFP tagged lamin B1, from which we can acquire both the DNA dye channel (nucleus) and the lamin B1 channel (nuclear envelope). Lamin B1 signals clearly mark the nuclei boundary and can thus serve as the real truth. We demonstrate that training a deep learning-based nuclei instance segmentation model with biological ground truth and manual annotations will result in significant differences in various metrics, such as volume or application-specific measurements. Also, we show the universalness of such issues with manual annotations by testing different state-of-the-art deep learning-based methods. We hope our work can raise within the biomedical image analysis community the awareness of (1) the importance of interdisciplinary collaborations, e.g., computational-experimental co-design for biological ground truth collection, and (2) potentially significant issues with manual annotation in training or evaluating deep learning-based segmentation models.