HUGNet: Hemi-Spherical Update Graph Neural Network Applied to Low-Latency Event-Based Optical Flow

Thomas Dalgaty, Thomas Mesquida, Damien Joubert, Amos Sironi, Pascal Vivet, Christoph Posch; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 3953-3962

Event camera pixels asynchronously output binary events corresponding to local light intensity changes in time. While encoding visual information in this fashion increases sparsity and the temporal detail of motion with respect to frame-based cameras, there is not yet an established machine learning method capable of exploiting these features to increase efficiency, reduce latency and, ultimately, perform optimally in event-based tasks. Graph neural networks are a promising avenue for such a method, but current solutions are too slow to be compatible with the continuous streaming nature of event-data. In this study, we propose a hemi-spherical update event-graph neural network that significantly reduces the complexity and latency of graph updating and event-level prediction. We compare our approach to existing graph neural network methods, as well as to dense-frame convolutional neural networks, on optical flow estimation tasks. Relative to the previous state of the art in event-graphs, we reduce event-graph update latency by more than four orders of magnitude and reduce the number of neural network calculations per second by 70x while predicting optical flow more accurately.