Automatic Fish Age Prediction using Deep Machine Learning: Combining Otolith Image NIR Spectra and Metadata Features

Aotian Zheng, Jenq-Neng Hwang, Yudong Liu, Qiancheng Li, Beverly Barnett, Farron Wallace, Irina Benson, Thomas Helser; Proceedings of the Winter Conference on Applications of Computer Vision (WACV) Workshops, 2025, pp. 1512-1519

Fish age is a crucial parameter for effective stock management influencing the estimation of growth rates mortality age at maturity and population trends. However traditional methods for determining fish age through otolith analysis are labor-intensive and prone to low repeatability. Recently Fourier transform near-infrared (FT-NIR) spectroscopy has emerged as a promising tool for more efficient age estimation. In this study we explore an alternative approach using RGB imagery of whole fish otoliths for rapid and accurate age determination. Specifically we evaluate the effectiveness of three data modalities--otolith images FT-NIR spectra and associated biological and geospatial data--in predicting fish age using deep learning techniques both independently and in combination. Drawing inspiration from generative AI which integrates diverse input modalities (e.g. image text masks) we propose two distinct methods for merging these data types beyond simple concatenation of feature embeddings. Our experiments conducted on otolith data from two commercially significant fish species--walleye pollock and red snapper--reveal that combining all three modalities yields the most accurate age predictions. Additionally we demonstrate that conditioning the image feature extraction process on both spectral and metadata enhances model performance.