Towards Virtual H&E Staining of Hyperspectral Lung Histology Images Using Conditional Generative Adversarial Networks

Neslihan Bayramoglu, Mika Kaakinen, Lauri Eklund, Janne Heikkila; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 64-71

The microscopic image of a specimen in the absence of staining appears colorless and textureless. Therefore, microscopic inspection of tissue requires chemical staining to create contrast. Hematoxylin and eosin (H&E) is the most widely used chemical staining technique in histopathology. However, such staining creates obstacles for automated image analysis systems. Due to different chemical formulations, different scanners, section thickness, and lab protocols, similar tissues can greatly differ in appearance. This huge variability is one of the main challenges in designing robust and resilient automated image analysis systems. Moreover, staining process is time consuming and its chemical effects deform structures of specimens. In this work, we develop a method to virtually stain unstained specimens. Our method utilizes dimension reduction and conditional adversarial generative networks (cGANs) which build highly non-linear mappings between input and output images. Conditional GANs ability to handle very complex functions and high dimensional data enables transforming unstained hyperspectral tissue image to their H&E equivalent which comprises highly diversified appearance. In the long term, such virtual digital H&E staining could automate some of the tasks in the diagnostic pathology workflow which could be used to speed up the sample processing time, reduce costs, prevent adverse effects of chemical stains on tissue specimens, reduce observer variability, and increase objectivity in disease diagnosis.