The GRODE Metrics: Exploring the Performance of Group Detection Approaches

Francesco Setti, Marco Cristani; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2015, pp. 36-42

The detection of groups of people is attracting the attention of many researchers in diverse fields, with a growing number of approaches published each year; despite this, the evaluation metrics are not consolidated, with some measures inherited from the people detection fields, other ones designed specifically for a particular approach, generating a set of not comparable figure of merits. Moreover, existent methods of analysis are scarcely expressive, for example ignoring the fact that groups have different cardinalities, and that obviously larger groups are harder to find. This paper fills this gap presenting GRODE, a suite of measures of accuracy which defines precision and recall on the groups, including the group cardinality as variable. This gives the possibility to investigate aspects never considered so far, such as the tendency of a method of over- or undersegmenting groups, or of better dealing with specific group cardinalities. The metrics have been applied to all the publicly available approaches of group detection, discovering interesting strength and pitfalls of the current literature, and promoting further research in the field.