Exploring Inter-Observer Differences in First-Person Object Views Using Deep Learning Models

Sven Bambach, Zehua Zhang, David J. Crandall, Chen Yu; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2773-2782

Recent advances in wearable camera technology have led many cognitive psychologists to study the development of the human visual system by recording the field of view of infants and toddlers. Meanwhile, the vast success of deep learning in computer vision is driving researchers in both disciplines to aim to benefit from each other's understanding. Towards this goal, we set out to explore how deep learning models could be used to gain developmentally relevant insight from such first-person data. We consider a dataset of first-person videos from different people freely interacting with a set of toy objects, and train different object-recognition models based on each subject's view. We observe large inter-observer differences and find that subjects who created more diverse images of an object result in models that learn more robust object representations.