SeeEEG: Semantic-aware EEG-based Multi-Modal Retrieval-Augmented Generation for High-Fidelity Visual Brain Decoding

Jun-Mo Kim, Woohyeok Choi, Sang-Jun Park, Keun-Soo Heo, Young-Han Son, Ji-Hye Oh, Dong-Hee Shin, Tae-Eui Kam; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 4883-4892

Visual brain decoding aims to understand how humans interpret visual stimuli and to reconstruct the perceived stimuli from brain signals. Electroencephalography (EEG) has emerged as a practical neuroimaging method for real-world applications due to its high portability, low cost, and feasibility. However, reconstructing visual stimuli from EEG remains challenging due to its limited spatial resolution, which hinders the capture of visual semantics and the generation of high-fidelity images. To address these challenges, we propose SeeEEG, an EEG-based retrieval-augmented generation framework for visual perception decoding. Firstly, we introduce a Semantic Region-aware Transformer (SRT) designed to aggregate EEG embeddings at both the electrode and regional levels, maximizing the utilization of spatial information despite EEG's limited spatial resolution. Next, we align EEG embeddings with image and text embeddings, respectively, using contrastive learning to ensure semantic consistency. Then we uses these aligned EEG embeddings to retrieve similar images and text with their pairs from an external image-text database. The EEG embeddings are augmented with retrieved samples via cross attention, enriching their high-level semantics and serving as guidance for a diffusion model to generate high-fidelity images. Experimental results demonstrate that SeeEEG outperforms state-of-the-art EEG-based methods in retrieval and image generation tasks, highlighting its effectiveness in capturing high-level semantics from EEG. These findings underscore the potential of SeeEEG as a robust framework for advancing EEG-based visual brain decoding.