Layer Optimized Spatial Spectral Masked Autoencoder for Semantic Segmentation of Hyperspectral Imagery

Aaron Perez, Saurabh Prasad; Proceedings of the Winter Conference on Applications of Computer Vision (WACV) Workshops, 2025, pp. 599-607

Hyperspectral imaging (HSI) captures detailed spectral data across numerous contiguous bands offering critical insights for applications such as environmental monitoring agriculture and urban planning. However the high dimensionality of HSI data poses significant challenges for traditional deep learning models necessitating more efficient solutions. In this paper we propose the Layer-Optimized Spatial-Spectral Transformer (LO-SST) a refined version of the Spatial-Spectral Transformer (SST) that incorporates structured layer pruning to reduce computational complexity while maintaining robust performance. LO-SST leverages self-supervised pretraining with a Masked Autoencoder (MAE) framework enabling the model to effectively learn spatial and spectral dependencies even in scenarios with limited labeled data. The use of separate spatial and spectral positional embeddings further enhances the model's ability to capture intricate relationships within hyperspectral data. Our experiments show that LO-SST achieves competitive segmentation accuracy while significantly reducing computational demands compared to traditional models. The effectiveness of random masking over alternative strategies during pretraining is also demonstrated underscoring its ability to preserve critical image features. These results highlight the potential of LO-SST as an efficient and scalable solution for hyperspectral image segmentation particularly in resource-constrained applications.