Cross-modal applications of a neuromorphic olfactory learning algorithm

Abstract

We adapted an olfactory neuromorphic algorithm to image and sound recognition. To achieve this, we carried out specific preprocessing procedures that were tailored to each modality. For images, we used the NIST digits dataset directly. For sound, we used samples from the Google Speech Command dataset. A gammatone filter was applied to each to reduce the noise of the short audio sample and convert the temporal sound signal to a positional frequency signal. The single stimulus test algorithm was then modified to handle audio processing on extracted columns from a gammatone filter spectrogram obtained from the sound file. We also implemented PCA for all modalities, retaining around 90% of the variance. The results showed that over sequential "olfactory" gamma cycles, the algorithm successfully achieved one-shot online learning over the image and sound modalities as well. However, PCA representations did not attain high similarities to their corresponding templates for all three modalities.