SmartOverlays: A Visual Saliency Driven Label Placement for Intelligent Human-Computer Interfaces

Srinidhi Hegde, Jitender Maurya, Ramya Hebbalaguppe; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 0-0

Abstract


In augmented reality (AR), the computer generated labels assist in understanding a scene by addition of contextual information. However, naive label placement often results in clutter and occlusion impairing the effectiveness of AR visualization. For label placement, the main objectives to be satisfied are, non occlusion to scene of interest, the proximity of labels to the object, and, temporally coherent labels in a video/live feed. We present a novel method for the placement of labels corresponding to objects of interest in a video/live feed that satisfies the aforementioned objectives. Our proposed framework, SmartOverlays, first identifies the objects and generates corresponding labels using a YOLOv2 in a video frame; at the same time, Saliency Attention Model (SAM) learns eye fixation points that aid in predicting saliency maps for label placement; finally, computes Voronoi partitions of the video frame, choosing the centroids of objects as seed points, to place labels for satisfying the proximity constraints with the object of interest. In addition, our approach incorporates tracking the detected objects in a frame to facilitate temporal coherence between frames that enhances readability of labels. We measure the effectiveness of SmartOverlays framework using two objective metrics: (a) Label Occlusion over Saliency (LOS), and, (b) temporal jitter metric to quantify jitter in the label placement.

Related Material


[pdf]
[bibtex]
@InProceedings{Hegde_2019_ICCV,
author = {Hegde, Srinidhi and Maurya, Jitender and Hebbalaguppe, Ramya},
title = {SmartOverlays: A Visual Saliency Driven Label Placement for Intelligent Human-Computer Interfaces},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
month = {Oct},
year = {2019}
}