Direct Generation of Images from EEG using Schrödinger Bridge

Abstract

Real-world data is often noisy, making it challenging to extract true signals Non-invasively recorded neural activities are among the most difficult data, yet its precise signal reconstruction is highly anticipated by communities developing non-invasive brain-machine interfaces. Several noise sources contribute to this challenge, including unrelated neuronal activity, non-brain bioelectricity, attenuation by the skull and scalp, and environmental noises. Additionally, the accumulation of noise varies significantly across subjects and recording sessions, resulting in widely diverging distributions of degraded observations. In this study, we propose modeling the noise accumulation process as a Schrodinger bridge and decoding the true signal by reversing this process. Compared to conventional guided Diffusion approach, our Schrodinger bridge approach effectively models diverse noise processes within a single framework, exhibiting greater robustness to inter-subject variability. Also, our approach doesnt require pre-aligning brain and image representations, which is an additional compute cost in the conventional approach.