PS-ARM: An End-to-End Attention-aware Relation Mixer Network for Person Search

Mustansar Fiaz, Hisham Cholakkal, Sanath Narayan, Rao Muhammad Anwer, Fahad Shahbaz Khan; Proceedings of the Asian Conference on Computer Vision (ACCV), 2022, pp. 3828-3844

Person search is a challenging problem with various real-world applications, that aims at joint person detection and re-identification of a query person from uncropped gallery images. Although, previous study focuses on rich feature information learning, it's still hard to retrieve the query person due to the occurrence of appearance deformations and background distractors. In this paper, we propose a novel person search network, which exploits the global relation between different local regions within RoI of a person. Our design focuses on the introduction of a novel attention-aware relation mixer (ARM) module containing a relation mixer block and a spatio-channel attention layer. The relation mixer block introduces a spatially attended spatial mixing and a channel-wise attended channel mixing for effectively capturing discriminative relation features within an RoI. These discriminative relation features are further enriched by introducing a spatio-channel attention where the foreground and background features are delineated in a joint spatio-channel space. Our ARM module is generic and it does not rely on fine-grained supervisions or topological assumptions, hence being easily integrated into any Faster R-CNN-based person search methods. Comprehensive experiments are performed on two challenging benchmark datasets: CUHK-SYSU [1] and PRW [2]. Our PS-ARM achieves state-of-the-art performance on both datasets. On the challenging PRW dataset, our PS-ARM achieves an absolute gain of 5% in the mAP score over SeqNet, while operating at a comparable speed. Our source code and trained models will be made public.