Instance-Level Meta Normalization

Songhao Jia, Ding-Jie Chen, Hwann-Tzong Chen; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 4865-4873


This paper presents a normalization mechanism called Instance-Level Meta Normalization (ILM Norm) to address a learning-to-normalize problem. ILM Norm learns to predict the normalization parameters via both the feature feed-forward and the gradient back-propagation paths. ILM Norm provides a meta normalization mechanism and has several good properties. It can be easily plugged into existing instance-level normalization schemes such as Instance Normalization, Layer Normalization, or Group Normalization. ILM Norm normalizes each instance individually and therefore maintains high performance even when small mini-batch is used. The experimental results show that ILM Norm well adapts to different network architectures and tasks, and it consistently improves the performance of the original models.

Related Material

author = {Jia, Songhao and Chen, Ding-Jie and Chen, Hwann-Tzong},
title = {Instance-Level Meta Normalization},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}