Comparing Representations for Event Camera-based Visual Object Tracking

Oussama Abdul Hay, Sara Alansari, Mohamad Alansari, Yahya Zweiri; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 4627-4636

Visual object tracking (VOT) in dynamic environments is challenging due to Motion Blur (MB), Illumination Variations (IV), and Fast Motion (FM), conditions where traditional RGB-based trackers often fail. Event cameras offer high temporal resolution and low latency, making them well-suited for such scenarios. However, current event-based tracking methods rely on arbitrarily selected event representations, lacking systematic evaluation. In this work, we benchmark five common representations, Event Frame (EF), Voxel Grid (VG), Pseudo-Frames (PS), Image of Warped Events (IWE), and Event Spike Tensor (EST), across two datasets (VisEvent and LaSOT), using both pure event and hybrid RGB-event trackers. We find that representation choice significantly impacts performance, with EST and IWE consistently outperforming others, while EF and PS show poor robustness under distribution shifts. To address representation variability, we propose the Gradient-Unified Shared Embedding Module (GUSEM), a dual-pathway architecture that fuses heterogeneous event inputs into a shared, edge-aware embedding space. GUSEM leverages spatial gradients for structural consistency and low-rank reconstruction for modality-specific semantics. Extensive experiments show that GUSEM improves accuracy and generalization across trackers, representations, and training regimes, establishing it as a robust, representation-agnostic solution for event-based tracking.