RecycleNet: Latent Feature Recycling Leads to Iterative Decision Refinement

Gregor Köhler, Tassilo Wald, Constantin Ulrich, David Zimmerer, Paul F. Jäger, Jörg K.H. Franke, Simon Kohl, Fabian Isensee, Klaus H. Maier-Hein; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024, pp. 810-818


Despite the remarkable success of deep learning systems over the last decade, a key difference still remains between neural network and human decision-making: As humans, we can not only form a decision on the spot, but also ponder, revisiting an initial guess from different angles, distilling relevant information, arriving at a better decision. Here, we propose RecycleNet, a latent feature recycling method, instilling the pondering capability for neural networks to refine initial decisions over a number of recycling steps, where outputs are fed back into earlier network layers in an iterative fashion. This approach makes minimal assumptions about the neural network architecture and thus can be implemented in a wide variety of contexts. Using medical image segmentation as the evaluation environment, we show that latent feature recycling enables the network to iteratively refine initial predictions even beyond the iterations seen during training, converging towards an improved decision. We evaluate this across a variety of segmentation benchmarks and show consistent improvements even compared with top-performing segmentation methods. This allows trading increased computation time for improved performance, which can be beneficial, especially for safety-critical applications.

Related Material

@InProceedings{Kohler_2024_WACV, author = {K\"ohler, Gregor and Wald, Tassilo and Ulrich, Constantin and Zimmerer, David and J\"ager, Paul F. and Franke, J\"org K.H. and Kohl, Simon and Isensee, Fabian and Maier-Hein, Klaus H.}, title = {RecycleNet: Latent Feature Recycling Leads to Iterative Decision Refinement}, booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, month = {January}, year = {2024}, pages = {810-818} }