Find My Astronaut Photo: Automated Localization and Georectification of Astronaut Photography

Alex Stoken, Kenton Fisher; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 6196-6205

Astronaut photography from the International Space Station (ISS) forms one of the longest continuous remote sensing datasets of Earth and has facilitated a large body of research ranging from glacial surface area analysis to volcanic sediment delivery. Such studies are enabled by the geolocation and georectification of the imagery. Yet, localizing astronaut photography of Earth is a challenging and labor-intensive task, tempering the amount of research that can be performed. We present a method for automatically localizing these images named Find My Astronaut Photo, which makes this task feasible by casting the problem as a precision-oriented image similarity and matching exercise. As the ISS orbits the globe, astronauts can view and photograph most locations on Earth, so there is no precomputable database of finite landmarks for image comparison. Therefore, we iteratively generate potentially similar images from geolocated satellite imagery on-demand and rely on an image matcher to discriminately detect overall similarity between these images and an astronaut photo. We evaluate various image matching techniques to find methods which allow us to discretize and reduce our search space to a manageable size, and locate astronaut photographs with high precision and speed. Find My Astronaut Photo has successfully geolocated over 30,000 photos to date, adding critical location information that increases the downstream utility of the Gateway to Astronaut Photography of Earth (GAPE) database. We also introduce AIMS, the Astronaut Imagery Matching Subset, a new real world evaluation dataset that joins the collection of challenging image matching benchmarks.