An Investigation of Attention Mechanisms in Histopathology Whole-Slide-Image Analysis for Regression Objectives

Philippe Weitz, Yinxi Wang, Johan Hartman, Mattias Rantalainen; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 611-619

Analysis of whole-slide-images (WSIs) of histopathology tissue sections remains challenging due to the gigapixel scale of these images, which often necessitates their division into smaller image tiles. Recently, attention mechanisms have been successfully applied to alleviate the tile-to-slide challenges for classification tasks based on WSIs. In this study, we explore the potential of attention mechanisms in regression settings, by comparing four modelling approaches, two of which use attention mechanisms. We evaluate these models both in a simulated experiment using the MNIST data set, and in real histopathology data sets focused on prediction of gene expression levels from WSIs, including an analysis of the local prediction performance using spatial transcriptomics. The MNIST simulation demonstrates that if only a small proportion of instances in a set of images contribute to the set-level regression label, attention mechanisms may be preferable to commonly applied weakly supervised models. When predicting gene expression from WSIs, the differences in performance between the models that we investigated were small. Nevertheless, we found some evidence that attention mechanisms may be more sensitive to domain shifts. In the regression-based task of gene expression prediction, the prediction performance in the present study appears to be limited by other factors rather than by the choice of modelling approach. Nevertheless, attention mechanisms appear promising for regression objectives and warrant further investigation.