Statistically-motivated Second-order Pooling

Kaicheng Yu, Mathieu Salzmann ; Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 600-616


However, the nature of such operations is usually computationally expensive, and resulting vector representation orders of magnitude larger than first-order baselines. Here, by contrast, we introduce a statistically-motivated framework that projects the second-order descriptor into a compact vector while improving the representational power. To this end, we design a parametric vectorization layer, which maps a covariance matrix, known to follow a Wishart distribution, to a vector whose elements can be shown to follow a Chi-square distribution. We then propose to make use of a square-root normalization, which makes the distribution of the resulting representation converge to a Gaussian, with which most classifiers of recent first-order networks complying. As evidenced by our experiments, this lets us outperform the state-of-the-art first-order and second-order models on several benchmark recognition datasets.

Related Material

[pdf] [arXiv]
author = {Yu, Kaicheng and Salzmann, Mathieu},
title = {Statistically-motivated Second-order Pooling},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
month = {September},
year = {2018}