EpO-Net: Exploiting Geometric Constraints on Dense Trajectories for Motion Saliency

Ijaz Akhter, Mohsen Ali, Muhammad Faisal, RICHARD HARTLEY; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 1884-1893

Abstract


The existing approaches for salient motion segmentation are unable to explicitly learn geometric cues and often give false detections on prominent static objects. We exploit multiview geometric constraints to avoid such shortcomings. To handle the nonrigid background like a sea, we also propose a robust fusion mechanism between motion and appearance-based features. We find dense trajectories, covering every pixel in the video, and propose trajectory-based epipolar distances to distinguish between background and foreground regions. Trajectory epipolar distances are data-independent and can be readily computed given a few features' correspondences between the images. We show that by combining epipolar distances with optical flow, a powerful motion network can be learned. Enabling the network to leverage both of these features, we propose a simple mechanism, we call input-dropout. Comparing the motion-only networks, we outperform the previous state of the art on DAVIS-2016 dataset by 5.2% in the mean IoU score. By robustly fusing our motion network with an appearance network using the input-dropout mechanism, we also outperform the previous methods on DAVIS-2016, 2017 and Segtrackv2 dataset.

Related Material


[pdf] [supp] [video]
[bibtex]
@InProceedings{Akhter_2020_WACV,
author = {Akhter, Ijaz and Ali, Mohsen and Faisal, Muhammad and HARTLEY, RICHARD},
title = {EpO-Net: Exploiting Geometric Constraints on Dense Trajectories for Motion Saliency},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
month = {March},
year = {2020}
}