Biology subjects

Sugano, H.

Publications and source records attributed to Sugano, H..

6 recordsLinked to original sources

bioRxiv2024

Speech Synthesis from Electrocorticogram During Imagined Speech Using a Transformer-Based Decoder and Pretrained Vocoder

Synthesizing speech from Electrocorticogram (ECoG) signals recorded during imagined speech remains a challenge due to the absence of synchronized audio signals for training. To address this, we propose a training framework that utilizes audio recorded during overt speech tasks as a surrogate ground truth for imagined speech signals, based on the consistency of the linguistic content. We employed a Transformer-based decoder to generate log-mel spectrograms from imagined speech ECoG, which were then converted into waveform audio using a pre-trained Parallel WaveGAN. In experiments involving ECoG recordings from 13 participants, the synthesized speech achieved dynamic time warping-aligned Pearson correlation coefficients ranging from 0.74 to 0.84 with the proxy targets. These results demonstrate that overt speech audio can serve as an effective training target for reconstructing imagined speech, offering a viable solution for training decoders in the absence of behavioral output.

neuroscience↗

bioRxiv2024

Image retrieval based on closed-loop visual-semantic neural decoding

Neural decoding via the latent space of deep neural network models can infer perceived and imagined images from neural activities, even when the image is novel for the subject and decoder. Brain-computer interfaces (BCIs) using the latent space enable a subject to retrieve intended image from a large dataset on the basis of their neural activities but have not yet been realized. Here, we used neural decoding in a closed-loop condition to retrieve images of the instructed categories from 2.3 million images on the basis of the latent vector inferred from electrocorticographic signals of visual cortices. Using a latent space of contrastive language-image pretraining (CLIP) model, two subjects retrieved images with significant accuracy exceeding 80% for two instructions. In contrast, the image retrieval failed using the latent space of another model, AlexNet. In another task to imagine an image while viewing a different image, the imagery made the inferred latent vector significantly closer to the vector of the imagined category in the CLIP latent space but significantly further away in the AlexNet latent space, although the same electrocorticographic signals from nine subjects were decoded. Humans can retrieve the intended information via a closed-loop BCI with an appropriate latent space.

neuroscience↗

bioRxiv2024

Feasibility of decoding covert speech in ECoG with aTransformer trained on overt speech

Several attempts for speech brain-computer interfacing (BCI) have been made to decode phonemes, sub-words, words, or sentences using invasive measurements, such as the electrocorticogram (ECoG), during auditory speech perception, overt speech, or imagined (covert) speech. Decoding sentences from covert speech is a challenging task. Sixteen epilepsy patients with intracranially implanted electrodes participated in this study, and ECoGs were recorded during overt speech and covert speech of eight Japanese sentences, each consisting of three tokens. In particular, Transformer neural network model was applied to decode text sentences from covert speech, which was trained using ECoGs obtained during overt speech. We first examined the proposed Transformer model using the same task for training and testing, and then evaluated the models performance when trained with overt task for decoding covert speech. The Transformer model trained on covert speech achieved an average token error rate (TER) of 46.6% for decoding covert speech, whereas the model trained on overt speech achieved a TER of 46.3% (p > 0.05; d = 0.07). Therefore, the challenge of collecting training data for covert speech can be addressed using overt speech. The performance of covert speech can improve by employing several overt speeches.

neuroscience↗

bioRxiv2021

Epileptic Spike Detection by Recurrent Neural Networks with Self-Attention Mechanism

ObjectiveBecause of the lack of highly skilled experts, automated technologies that support electroencephalogram (EEG)-based in epilepsy diagnosis are advancing. Deep convolutional neural network-based models have been used successfully for detecting epileptic spikes, one of the biomarkers, from EEG. However, a sizeable number of supervised EEG records are required for training. ApproachThis study introduces the Satelight model, which uses the self-attention (SA) mechanism. The model was trained using a clinical EEG dataset labeled by five specialists, including 16,008 epileptic spikes and 15,478 artifacts from 50 children. The SA mechanism is expected to reduce the number of parameters and efficiently extract features from a small amount of EEG data. To validate the effectiveness, we compared various spike detection approaches with the clinical EEG data. Main resultsThe experimental results showed that the proposed method detected epileptic spikes more effectively than other models (accuracy = 0.876 and false positive rate = 0.133). SignificanceThe proposed model had only one-tenth the number of parameters as the other effective model, despite having such a high detection performance. Further exploration of the hidden parameters revealed that the model automatically attended to the EEGs characteristic waveform locations of interest.

bioengineering↗

bioRxiv2020

Epileptic Spike Detection by Using a Linear-Phase Convolutional Neural Network

To cope with the lack of highly skilled professionals, machine learning with proper signal processing is key for establishing automated diagnostic-aid technologies with which to conduct epileptic electroencephalogram (EEG) testing. In particular, frequency filtering with the appropriate passbands is essential for enhancing the biomarkers--such as epileptic spike waves--that are noted in the EEG. This paper introduces a novel class of neural networks (NNs) that have a bank of linear-phase finite impulse response filters at the first layer as a preprocessor that can behave as bandpass filters that extract biomarkers without destroying waveforms because of a linear-phase condition. Besides, the parameters of the filters are also data-driven. The proposed NNs were trained with a large amount of clinical EEG data, including 15,833 epileptic spike waveforms recorded from 50 patients, and their labels were annotated by specialists. In the experiments, we compared three scenarios for the first layer: no preprocessing, discrete wavelet transform, and the proposed data-driven filters. The experimental results show that the trained data-driven filter bank with supervised learning behaves like multiple bandpass filters. In particular, the trained filter passed a frequency band of approximately 10-30 Hz. Moreover, the proposed method detected epileptic spikes, with the area under the receiver operating characteristic curve of 0.967 in the mean of 50 intersubject validations.

neuroscience↗

bioRxiv2020

Multiband entropy-based feature-extraction method for automatic identification of epileptic focus based on high- frequency components in interictal iEEG

Presurgical investigations for categorizing focal patterns are crucial, leading to localization and surgical removal of the epileptic focus. This paper presents a machine learning approach using information theoretic features extracted from high-frequency subbands to detect the epileptic focus from interictal intracranial electroencephalogram (iEEG). It is known that high-frequency subbands (>80 Hz) include important biomarkers such as high-frequency oscillations (HFOs) for identifying epileptic focus commonly referred to as the seizure on-set zone (SOZ). In this analysis, the multi-channel interictal iEEG signals were splitted into segments and each segment was decomposed into multiple high-frequency subbands. The different types of entropy were calculated for each of the subbands and the sparse linear discriminant analysis (sLDA) was applied to select the prominent entropy features. Due to the imbalance of SOZ and non-SOZ channels in iEEG data, the use of machine learning techniques is always tricky. To deal with the imbalanced learning problem, an adaptive synthetic oversampling approach (ADASYN) with radial basis function kernel-based SVM was used to detect the focal segments. Finally, the epileptic focus was identified based on detection of focal segments on SOZ and non-SOZ channels. Eight patients were examined to observe the efficiency of the automatic detector. The experimental results and statistical tests indicate that the proposed automatic detector can identify the epileptic focus accurately and efficiently.

bioengineering↗