Silent Speech Recognition with Wearable Magnetometers

Abstract

Next-generation human-computer interaction (HCI) is moving towards more seamless, intuitive, and personal modes of communication, redefining how we interact with technology and one another. Within this landscape, silent speech recognition (SSR) offers a powerful new interaction paradigm, enabling hands-free, private interaction while supporting individuals with speech impairments and enabling communication in noisy or sensitive environments. Recent advances in miniaturized sensors and artificial intelligence (AI) have accelerated the development of more sophisticated wearable SSR systems, driven by growing demand for effortless and accessible communication. Although electrophysiological (ExG) modalities, particularly electromyography (EMG), have dominated early efforts in developing wearable SSR, critical challenges remain. Limited generalizability across users, sensor-skin interface issues, and difficulties with the comfort of use are all current roadblocks to reliable, high-fidelity signals in a wearable form factor. We propose that magnetometers offer a promising alternative to ExG and have the potential to unlock more robust, generalizable, and user-friendly SSR systems. We demonstrate that magnetometers embedded in a headphone form factor achieve a per-user SSR accuracy of 86%, significantly outperforming previously reported state-of-the-art wearable headphones combining ExG and inertial measurement units (IMUs). In addition, we show that wearable magnetometry enables generalization across individuals for SSR. Extending beyond headphones, we also introduce a necklace form factor with magnetometers that is capable of decoding both silent and overt speech in ambient conditions, further showcasing the versatility of magnetometers across different wearable designs in real-world conditions.