Partial Texture VAE: Color and Texture Encoder for Rock Particle Images

Tetsushi Yamada, Simone Di Santo; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 4999-5008

We propose Partial Texture VAE (PT-VAE) for rock particle image analysis a variant of the variational autoencoder (VAE) specialized in encoding color and texture properties from arbitrarily sized and shaped unresized images containing invalid background pixels into the vectors with the same size. PT-VAE integrates partial convolution and the mask aware feature aggregation layer in the encoder and is trained by minimizing a simplified style loss in an unsupervised manner. Input texture images do not require resizing or cropping thus preventing deformation and under-representation of the overall texture pattern. This consideration is often overlooked in data-driven texture analysis. Experiments using our textured rock particle images show that the encoded features efficiently capture color and texture information and are invariant to size and shape of the image as well as invalid pixels. Image retrieval tests show that the PT-VAE incorporating the mask aware Texture Encoding Layer (TEL) outperformed other configurations and existing methods.