Spk2ImgMamba: Spiking Camera Image Reconstruction with Multi-Scale State Space Models

Jiaoyang Yin, Bin Fan, Chao Xu, Tiejun Huang, Boxin Shi; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 9324-9334

As a bio-inspired vision sensor the spiking camera has showcased remarkable capability in high-speed imaging with a sampling rate of 40000 Hz. Reconstructing clear images from continuous spike streams which is obtained by each photosensor continuously detecting photons and firing them asynchronously has garnered significant attention. Despite promising results existing spike-to-image reconstruction methods face challenges in balancing global receptive fields and efficient computation due to the inherent limitations of their backbones. Recently due to powerful long-range modeling and linear complexity the state space model (SSM) has emerged as a competitive alternative to CNNs and Transformers. In this paper we propose a lightweight spike-to-image reconstruction network that harnesses Mamba as the backbone. Our approach sequentially executes three core modules: temporal information integration spatial feature enhancement and progressive image reconstruction. The former accumulates cues across diverse temporal windows to explore both long-term and short-term contexts. Subsequently to model global dependencies while heightening local detail perception we develop a multi-scale SSM block characterized by multi-scale multi-direction scanning which effectively boosts spatial feature representations. Finally intensity images are decoded progressively from the enhanced light-intensity features. Extensive experiments on both synthetic and real-captured data demonstrate that our approach achieves state-of-the-art performance with only 10% of the network parameters and nearly two orders of magnitude less computational effort. The code will be available at https://github.com/interstellarH/Spk2ImgMamba.