FootNet: An efficient convolutional network for multiview 3D foot reconstruction

Felix Kok, James Charles, Roberto Cipolla; Proceedings of the Asian Conference on Computer Vision (ACCV), 2020

Automatic biometric analysis of the human body is normally reserved for expensive customisation of clothing items e.g. for sports or medical purposes. These systems are usually built upon photogrammetric techniques currently requiring a rig and well calibrated cameras. Here we propose building on advancements in deep learning as well as utilising technology present in mobile phones for cheaply and accurately determining biometric data of the foot. The system is designed to run efficiently in a mobile phone app where it can be used in uncalibrated environments and without rigs. By scanning the foot with the phone camera, our system recovers both the 3D shape as well as the scale of the foot, opening the door way for automatic shoe size suggestion. Our contributions are (1) an efficient multiview feed forward neural network capable of inferring foot shape and scale, (2) a system for training from completely synthetic data and (3) a dataset of multiview feet images for evaluation. We fully ablate our system and show our design choices to improve performance at every stage. Our final design has a vertex error of only 1mm (for 25cm long synthetic feet) and 4mm error in foot length on real feet.