Low-Resolution-Only Microscopy Super-Resolution Models Generalizing to Non-Periodicities at Atomic Scale

Björn Möller, Zhengyang Li, Markus Etzkorn, Tim Fingscheidt; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 6936-6945

Super-resolution (SR) methods can accelerate microscopy image capturing and improve quality. Yet training data is often scarce and low in variability leading to overfitting models that fail to preserve unseen image structures. Therefore in this work we investigate SR model generalization in a low-resource domain here: material science. First we propose a training pipeline based on PixMix augmentation for microscopy SR using low-resolution only training data to generate pseudo LR/HR training pairs. The augmentation introduces variability into training images by blending them with high-detail out-of-domain images. Second using scanning transmission electron microscopy (STEM) images we show that our proposed training pipeline improves the SR model generalization for non-periodic high-resolution test data of crystalline atomic structures even if only periodic low-resolution data is used for training. Furthermore our proposed pipeline enables STEM SR models to generalize to images with noise characteristics from an unseen recording session. Third we investigate effects of mixing augmentation strength. Finally we validate the usage of PixMix on a more comprehensive STEM dataset. Our results demonstrate that frequent image mixing utilizing high-detail out-of-domain data improves SR generalization within low-resource domains such as atomic-scale STEM images of non-periodic matter. Data and code is available.