Non-Linear Convolution Filters for CNN-Based Learning

Georgios Zoumpourlis, Alexandros Doumanoglou, Nicholas Vretos, Petros Daras; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 4761-4769


During the last years, Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in image classification. Their architectures have largely drawn inspiration by models of the primate visual system. However, while recent research results of neuroscience prove the existence of non-linear operations in the response of complex visual cells, little effort has been devoted to extend the convolution technique to non-linear forms. Typical convolutional layers are linear systems, hence their expressiveness is limited. To overcome this, various non-linearities have been used as activation functions inside CNNs, while also many pooling strategies have been applied. We address the issue of developing a convolution method in the context of a computational model of the visual cortex, exploring quadratic forms through the Volterra kernels. Such forms, constituting a more rich function space, are used as approximations of the response profile of visual cells. Our proposed second-order convolution is tested on CIFAR-10 and CIFAR-100. We show that a network which combines linear and non-linear filters in its convolutional layers, can outperform networks that use standard linear filters with the same architecture, yielding results competitive with the state-of-the-art on these datasets.

Related Material

[pdf] [arXiv]
author = {Zoumpourlis, Georgios and Doumanoglou, Alexandros and Vretos, Nicholas and Daras, Petros},
title = {Non-Linear Convolution Filters for CNN-Based Learning},
booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
month = {Oct},
year = {2017}