Simple Baselines Can Fool 360deg Saliency Metrics

Yasser Abdelaziz Dahou Djilali, Kevin McGuinness, Noel E. O'Connor; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 3750-3756

Evaluating a model's capacity to predict human fixations in 360deg scenes is a challenging task. 360deg saliency requires different assumptions compared to 2D as a result of the way the saliency maps are collected and pre-processed to account for the difference in statistical bias (Equator vs Center bias). However, the same classical metrics from the 2D saliency literature are typically used to evaluate 360deg models. In this paper, we show that a simple constant predictor, i.e. the average map across Salient360 and Sitzman training sets can fool existing metrics and achieve results on par with specialized models. Thus, we propose a new probabilistic metric based on the independent Bernoullis assumption that is more suited to the 360deg saliency task.