On the Impact of Parallax Free Colour and Infrared Image Co-Registration to Fused Illumination Invariant Adaptive Background Modelling

Michael Loveday, Toby P. Breckon; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018, pp. 1186-1195

Contrary to other visible-band (colour, RGB) and infrared-band (T) cross-modal work in the field, we present a practical approach to parallax-free RGB-T image formation using a combination of optical engineering (beam-splitter) and visual geometry. We use background to foreground object separation, a task inherently susceptible to multi-view parallax issues, to illustrate our approach. We evaluate the complementary nature of visible and far infrared (thermal, long-wave) information through three fusion schemes which physically combine visible-band (colour, RGB) and infrared-band (T) imagery into a co-registered, parallax free RGB-T image model. The performance of this combined RGB-T image model is assessed against standalone colour and thermal imagery for object detection within an adaptive background modelling framework. Illumination invariant background models, incorporating additional infrared information, increase the accuracy and precision of foreground object detection by over 10% on average when compared to standalone visible-band and over 5% for standalone infrared. Furthermore, the use of combined colour and infrared within adaptive background modelling provides superior results under conditions when either visible or infrared band performance is notably degraded. Evaluation is performed over a range of challenging conditions, over which the combined use of infrared and illumination invariant colour emerges as a more robust background modelling approach.