Visual Transformation Aided Contrastive Learning for Video-Based Kinship Verification

Hamdi Dibeklioglu; The IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2459-2468

Abstract


Automatic kinship verification from facial information is a relatively new and open research problem in computer vision. This paper explores the possibility of learning an efficient facial representation for video-based kinship verification by exploiting the visual transformation between facial appearance of kin pairs. To this end, a Siamese-like coupled convolutional encoder-decoder network is proposed. To reveal resemblance patterns of kinship while discarding the similarity patterns that can also be observed between people who do not have a kin relationship, a novel contrastive loss function is defined in the visual appearance space. For further optimization, the learned representation is fine-tuned using a feature-based contrastive loss. An expression matching procedure is employed in the model to minimize the negative influence of expression differences between kin pairs. Each kin video is analyzed by a sliding temporal window to leverage short-term facial dynamics. The effectiveness of the proposed method is assessed on seven different kin relationships using smile videos of kin pairs. On the average, 93.65% verification accuracy is achieved, improving the state of the art.

Related Material


[pdf]
[bibtex]
@InProceedings{Dibeklioglu_2017_ICCV,
author = {Dibeklioglu, Hamdi},
title = {Visual Transformation Aided Contrastive Learning for Video-Based Kinship Verification},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {Oct},
year = {2017}
}