Biology subjects

Giordano, B. L.

Publications and source records attributed to Giordano, B. L..

10 recordsLinked to original sources

bioRxiv2026

A behaviourally normed database of 1,377 natural sounds for auditory cognition and neuroscience

Natural-sound research requires stimulus sets that combine acoustic standardization with detailed behavioural characterization. We present 1,377 two-second sounds representing 240 expert-defined source--action classes. We call this database "MaMa Sounds", as it resulted from the collaborative effort of two academic teams in Maastricht and Marseille. The sounds were manually curated, segmented, sampled at 16 kHz, and labelled with a noun identifying the source and a verb identifying the action. We release deidentified trial-level identification and familiarity data together with multiple per-sound norms (e.g., identification accuracy, confidence and agreement; familiarity), along with overall norms derived with principal component analysis. Noun, verb, and joint noun--verb norms are provided as direct means and medians with the number of contributing observations. This battery preserves process-specific information, while two principal-component scores provide compact overall behavioural-identifiability measures derived from response ease, semantic correspondence, agreement, and familiarity. The repository also contains deterministic response-cleaning code, participant and reference Word2Vec representations, and code reproducing the public sound-level tables. The resource supports stimulus selection, matching, and continuous modelling in auditory cognition and neuroscience.

neuroscience↗

bioRxiv2026

Multiscale Temporal Processing Supports Sound Recognition under Causal Constraints

The auditory system operates under a fundamental computational constraint: at any moment, it has access only to past and present acoustic information. At the same time, it processes sounds across multiple temporal scales, although the computational advantage of this organization remains unclear. Despite both being defining characteristics of biological auditory processing, causal processing and multiscale temporal processing are rarely considered together in computational models. Here, we introduce the Multiscale Convolutional Recurrent Neural Network (MSCRNN) architecture, a brain-inspired model designed to test whether processing sounds across multiple temporal contexts improves recognition under causal constraints. Multiscale processing substantially improved recognition under causal constraints, enabling performance comparable to non-causal architectures after an initial evidence-accumulation period while providing only limited benefit when future acoustic information was available. Analyses of the networks internal representations revealed a progressive transformation from stream-specific acoustic representations to increasingly integrated category-level representations across recurrent processing stages. Together, these findings provide a computational rationale for why biological auditory systems may benefit from multiscale temporal processing under causal constraints and establish the MSCRNN as an interpretable framework for generating and testing hypotheses about the temporal dynamics of auditory processing using electrophysiological and neuroimaging data.

neuroscience↗

bioRxiv2025

Coupled rhythms in early auditory cortex mirror speech acoustics

Theta and gamma neural dynamics dominate the human auditory cortex during speech perception and have been proposed to track syllable boundaries and encode phonemic information, respectively. To what extent these rhythms are intrinsically generated or imposed by speech acoustics remains unsolved. Applying analytic methods from neuroscience to speech audio corpora from 17 languages, we found that canonical brain features --theta, gamma, and their phase-amplitude coupling-- are a robust and specific acoustic signature of speech envelope across languages. They represent syllabic rate (2-6 Hz), vocalic features (30-50 Hz), and fundamental frequency (100-150 Hz). Intracerebral (sEEG) recordings from the auditory cortex of 18 epilepsy patients revealed that theta-gamma dynamics and their coupling are absent at rest. They emerge during speech perception and are linearly driven by the acoustic envelope, consistent with an evoked origin. Nevertheless, these responses originate from distinct yet functionally interconnected neural populations, indicating that the early auditory cortex demultiplexes speech timescales. Thus, early auditory cortex mirrors theta-gamma speech rhythms across segregated neural populations. IN BRIEFEarly auditory cortex demultiplexes speech: stimulus-locked gamma activity aligns theta phase with the nested acoustic structure, functionally organising speech timescales

neuroscience↗

bioRxiv2024

Bridging Auditory Perception and Natural Language Processing with Semantically informed Deep Neural Networks

Sound recognition is effortless for humans but poses a significant chal-lenge for artificial hearing systems. Deep neural networks (DNNs), especially convolutional neural networks (CNNs), have recently sur-passed traditional machine learning in sound classification. However, current DNNs map sounds to labels using binary categorical variables, neglecting the semantic relations between labels. Cognitive neuroscience research suggests that human listeners exploit such semantic informa-tion besides acoustic cues. Hence, our hypothesis is that incorporating semantic information improves DNNs sound recognition performance, emulating human behavior. In our approach, sound recognition is framed as a regression problem, with CNNs trained to map spec-trograms to continuous semantic representations from NLP models (Word2Vec, BERT, and CLAP text encoder). Two DNN types were trained: semDNN with continuous embeddings and catDNN with cat-egorical labels, both with a dataset extracted from a collection of 388,211 sounds enriched with semantic descriptions. Evaluations across four external datasets, confirmed the superiority of semantic labeling from semDNN compared to catDNN, preserving higher-level relations. Importantly, an analysis of human similarity ratings for natural sounds, showed that semDNN approximated human listener behavior better than catDNN, other DNNs, and NLP models. Our work contributes to understanding the role of semantics in sound recognition, bridging the gap between artificial systems and human auditory perception.

neuroscience↗

bioRxiv2024

Reconstructing Voice Identity from Noninvasive Auditory Cortex Recordings

The cerebral processing of voice information is known to engage, in human as well as non-human primates, "temporal voice areas" (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity--the voice latent space (VLS)--and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.

neuroscience↗

bioRxiv2021

Group-level inference of information-based measures for the analyses of cognitive brain networks from neurophysiological data

The reproducibility crisis in neuroimaging and in particular in the case of underpowered studies has introduced doubts on our ability to reproduce, replicate and generalize findings. As a response, we have seen the emergence of suggested guidelines and principles for neuroscientists known as Good Scientific Practice for conducting more reliable research. Still, every study remains almost unique in its combination of analytical and statistical approaches. While it is understandable considering the diversity of designs and brain data recording, it also represents a striking point against reproducibility. Here, we propose a non-parametric permutation-based statistical framework, primarily designed for neurophysiological data, in order to perform group-level inferences on non-negative measures of information encompassing metrics from information-theory, machine-learning or measures of distances. The framework supports both fixed- and random-effect models to adapt to inter-individuals and inter-sessions variability. Using numerical simulations, we compared the accuracy in ground-truth retrieving of both group models, such as test- and cluster-wise corrections for multiple comparisons. We then reproduced and extended existing results using both spatially uniform MEG and non-uniform intracranial neurophysiological data. We showed how the framework can be used to extract stereotypical task- and behavior-related effects across the population covering scales from the local level of brain regions, inter-areal functional connectivity to measures summarizing network properties. We also present an open-source Python toolbox called Frites1 that includes the proposed statistical pipeline using information-theoretic metrics such as single-trial functional connectivity estimations for the extraction of cognitive brain networks. Taken together, we believe that this framework deserves careful attention as its robustness and flexibility could be the starting point toward the uniformization of statistical approaches. Graphical abstract O_FIG_DISPLAY_L [Figure 1] M_FIG_DISPLAY C_FIG_DISPLAY HighlightsO_LIGroup-level statistics for extracting neurophysiological cognitive brain networks C_LIO_LICombining non-parametric permutations with measures of information C_LIO_LIFixed- and random-effect models, test- and cluster-wise corrections C_LIO_LIMulti-level inferences, from local regions to inter-areal functional connectivity C_LIO_LIA Python open-source toolbox called Frites includes the proposed statistical methods C_LI

neuroscience↗

bioRxiv2019

Decoding the sound of hand-object interactions in primary somatosensory cortex

Neurons, even in earliest sensory regions of cortex, are subject to a great deal of contextual influences from both within and across modality connections. Recent work has shown that primary sensory areas can respond to and in some cases discriminate stimuli not of their target modality: for example, primary somatosensory cortex (SI) discriminates visual images of graspable objects. In the present work, we investigated whether SI would discriminate sounds depicting hand-object interactions (e.g. bouncing a ball). In a rapid event-related functional magnetic resonance imaging (fMRI) experiment, participants listened attentively to sounds from three categories: hand-object interactions, and control categories of pure tones and animal vocalizations, while performing a one-back repetition detection task. Multi-voxel pattern analysis revealed significant decoding of different hand-object interactions within SI, but not for either control category. Crucially, in the hand-sensitive voxels defined from an independent tactile localizer, decoding accuracies were significantly higher for hand-object interactions compared to pure tones in left SI. Our findings indicate that simply hearing sounds depicting familiar hand-object interactions elicit different patterns of activity in SI, despite the complete absence of tactile stimulation. These results highlight the rich information that can be transmitted across sensory modalities even to primary sensory areas.

neuroscience↗

bioRxiv2018

Causal inference in the multisensory brain

When combining information across different senses humans need to flexibly select cues of a common origin whilst avoiding distraction from irrelevant inputs. The brain could solve this challenge using a hierarchical principle, by deriving rapidly a fused sensory estimate for computational expediency and, later and if required, filtering out irrelevant signals based on the inferred sensory cause(s). Analysing time- and source-resolved human magnetoencephalographic data we unveil a systematic spatio-temporal cascade of the relevant computations, starting with early segregated unisensory representations, continuing with sensory fusion in parietal-temporal regions and culminating as causal inference in the frontal lobe. Our results reconcile previous computational accounts of multisensory perception by showing that prefrontal cortex guides flexible integrative behaviour based on candidate representations established in sensory and association cortices, thereby framing multisensory integration in the generalised context of adaptive behaviour.

neuroscience↗

bioRxiv2018

From categories to dimensions: spatio-temporal dynamics of the cerebral representations of emotion in voice

Whether the human brain represents emotional stimuli as discrete categories or continuous dimensions is still widely debated. Here we directly contrasted the power of categorical and dimensional models at explaining behavior and cerebral activity in the context of perceived emotion in the voice. We combined functional magnetic resonance imaging (fMRI) and magneto-encephalography (MEG) to measure with high spatiotemporal precision the dynamics of cerebral activity in participants who listened to voice stimuli expressing a range of emotions. The participants also provided a detailed perceptual assessment of the stimuli. By using representational similarity analysis (RSA), we show that the participants perceptual representation of the stimuli was initially dominated by discrete categories and an early (<200ms) cerebral response. These responses showed significant associations between brain activity and the categorical model in the auditory cortex starting as early as 77ms. Furthermore, we observed strong associations between the arousal and valence dimensions and activity in several cortical and subcortical areas at later latencies (>500ms). Our results thus show that both categorical and dimensional models account for patterns of cerebral responses to emotions in voices but with a different timeline and detail as to how these patterns evolve from discrete categories to progressively refined continuous dimensions.\n\nOne Sentence Summary: Emotions expressed in the voice are instantly categorized in cortical processing and their distinct qualities are refined dimensionally only later on.

neuroscience↗

bioRxiv2016

Contributions of local speech encoding and functional connectivity to audio-visual speech integration

Seeing a speakers face enhances speech intelligibility in adverse environments. We investigated the underlying network mechanisms by quantifying local speech representations and directed connectivity in MEG data obtained while human participants listened to speech of varying acoustic SNR and visual context. During high acoustic SNR speech encoding by entrained brain activity was strong in temporal and inferior frontal cortex, while during low SNR strong entrainment emerged in premotor and superior frontal cortex. These changes in local encoding were accompanied by changes in directed connectivity along the ventral stream and the auditory-premotor axis. Importantly, the behavioural benefit arising from seeing the speaker's face was not predicted by changes in local encoding but rather by enhanced functional connectivity between temporal and inferior frontal cortex. Our results demonstrate a role of auditory-motor interactions in visual speech representations and suggest that functional connectivity along the ventral pathway facilitates speech comprehension in multisensory environments.

neuroscience↗