Color Representation in CNNs: Parallelisms With Biological Vision

Ivet Rafegas, Maria Vanrell; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2697-2705

CNNs trained for object recognition present representational capabilities approaching to primate visual systems. This provides a computational framework to explore how image features are efficiently represented. Here, we dissect a trained CNN to study how color is represented. We use a methodology used in physiology that is measuring index of selectivity of individual neurons to specific features. We use ImageNet Dataset images and synthetic versions of them to quantify color tuning properties of artificial neurons to provide a classification of the network population. We conclude three main levels of color representation showing parallelisms with biological visual systems: a decomposition in a circular hue space encoding single color regions with a wide hue sampling beyond the first layer (V2); opponent low-dimensional spaces in early stages (V1); a strong color-shape entanglement representing object-parts, object-shapes, or object-surrounds configurations in deeper layers (V4 or IT)