Biology subjects

Nygaard, L. C.

Publications and source records attributed to Nygaard, L. C..

6 recordsLinked to original sources

bioRxiv2024

Acoustic parameter combinations underlying mapping of auditory pseudoword sounds to multiple domains of meaning: a machine learning approach

In spoken language, iconicity, referring to the resemblance between the sound structure of words and their meaning, is often studied using pseudowords. Previously, we showed that representational dissimilarity matrices (RDMs) of the shape ratings of pseudowords correlated significantly with RDMs of acoustic parameters reflecting spectro-temporal variations; the ratings also correlated significantly with voice quality parameters. Here, we examined how perceptual ratings relate to these parameters of pseudowords across eight meaning domains. We largely replicated our previous findings for shape, while observing different patterns for other domains. Using a k-nearest-neighbor (KNN) machine-learning algorithm, we compared 4095 combinations of twelve acoustic parameters (3 spectro-temporal and 9 characterizing vocal quality) to determine the optimal combination associated with iconicity ratings in each domain. We found that iconic mappings were linked to domain-specific combinations of acoustic parameters. One spectro-temporal parameter, the fast Fourier transform, contributed to all domains, indicating the importance of time-varying spectral properties for iconicity judgments. We applied the KNN approach to generate shape ratings for 160 real words. These generated ratings strongly correlated with perceptual ratings of real words, indicating the value of the KNN approach to assess iconic mapping in natural languages. Our findings support the relevance of iconicity to language.

neuroscience↗

bioRxiv2024

DIFFERENTIAL CONTRIBUTIONS OF THE SPECTRO-TEMPORAL AND VOCAL CHARACTERISTICS OF AUDITORY PSEUDOWORDS TO MULTIPLE SOUND-SYMBOLIC MAPPINGS

In spoken language, iconic (sound-symbolic) words are those whose sounds convey their meaning. Iconicity is widespread in natural languages, whether signed or spoken, but its instantiation across different domains of meaning has not been systematically studied. Here, participants rated a set of 537 auditory pseudowords on opposing dimensions of eight different sound-symbolic domains: shape (rounded-pointed), roughness (smooth-rough), hardness (hard-soft), weight (light-heavy), size (small-big), brightness (bright-dark), arousal (calming-exciting), and valence (good-bad). Ratings showed cross-domain relationships, some mirroring those between corresponding physical domains, e.g. size and weight ratings were associated, reflecting a physical size-weight relationship, while others involved figurative relationships, e.g. bright/dark mapped onto good/bad, respectively. Using four separate multiple regression analyses, we found that the phonetic categories, phonemic associations, and acoustic associations at both the whole-item and segmental levels, formed unique sets with characteristic feature weightings for each meaning domain studied. For the majority (12 of 16 dimensions), the phonemic regression accounted for the most variance in the ratings, followed by the phonetic category, segmental acoustic and whole-item acoustic regressions. We conclude that pseudoword iconicity is present across a range of meaning domains, with domain-specific patterns of linguistic and acoustic properties. Linguistic characterizations best captured judgments of iconicity across domains, suggesting that these properties represent the bundles of acoustic, articulatory, and abstract linguistic factors that may underlie iconicity in spoken language.

neuroscience↗

bioRxiv2024

Phonetic underpinnings of sound symbolism across multiple domains of meaning

Sound symbolism occurs when the sound of a word alone can convey its meaning, e.g. balloon and spike sound rounded and pointed, respectively. Sound-symbolic correspondences are widespread in natural languages, but it is unclear how they are instantiated across different domains of meaning. Here, participants rated auditory pseudowords on opposing scales of seven different sound-symbolic domains: shape (rounded-pointed), texture (hard-soft), weight (light-heavy), size (small-big), brightness (bright-dark), arousal (calming-exciting), and valence (good-bad). Ratings showed cross-domain relationships, some mirroring those between corresponding physical domains, e.g. size and weight ratings were associated, reflecting a physical size-weight relationship, while others involved metaphorical mappings, e.g., bright/dark mapped onto good/bad, respectively. The phonetic features of the pseudowords formed unique sets with characteristic feature weightings for each domain and tended to follow the cross-domain ratings relationships. These results suggest that sound-symbolic correspondences rely on domain-specific patterns of phonetic features, with cross-domain correspondences reflecting physical or metaphorical relationships.

neuroscience↗

bioRxiv2023

NEURAL BASIS OF SOUND-SYMBOLIC PSEUDOWORD-SHAPE CORRESPONDENCES

Non-arbitrary mapping between the sound of a word and its meaning, termed sound symbolism, is commonly studied through crossmodal correspondences between sounds and visual shapes, e.g., auditory pseudowords, like mohloh and kehteh, are matched to rounded and pointed visual shapes, respectively. Here, we used functional magnetic resonance imaging (fMRI) during a crossmodal matching task to investigate the hypotheses that sound symbolism (1) involves language processing; (2) depends on multisensory integration; (3) reflects embodiment of speech in hand movements. These hypotheses lead to corresponding neuroanatomical predictions of crossmodal congruency effects in (1) the language network; (2) areas mediating multisensory processing, including visual and auditory cortex; (3) regions responsible for sensorimotor control of the hand and mouth. Right-handed participants (n = 22) encountered audiovisual stimuli comprising a simultaneously presented visual shape (rounded or pointed) and an auditory pseudoword ( mohloh or kehteh) and indicated via a right-hand keypress whether the stimuli matched or not. Reaction times were faster for congruent than incongruent stimuli. Univariate analysis showed that activity was greater for the congruent compared to the incongruent condition in the left primary and association auditory cortex, and left anterior fusiform/parahippocampal gyri. Multivoxel pattern analysis revealed higher classification accuracy for the audiovisual stimuli when congruent than when incongruent, in the pars opercularis of the left inferior frontal (Brocas area), the left supramarginal, and the right mid-occipital gyri. These findings, considered in relation to the neuroanatomical predictions, support the first two hypotheses and suggest that sound symbolism involves both language processing and multisensory integration. HIGHLIGHTSO_LIfMRI investigation of sound-symbolic correspondences between auditory pseudowords and visual shapes C_LIO_LIFaster reaction times for congruent than incongruent audiovisual stimuli C_LIO_LIGreater activation in auditory and visual cortices for congruent stimuli C_LIO_LIHigher classification accuracy for congruent stimuli in language and visual areas C_LIO_LISound symbolism involves language processing and multisensory integration C_LI

neuroscience↗

bioRxiv2019

Acoustic and visual stimulus parameters underlying sound-symbolic crossmodal correspondences

It is often assumed that a fundamental property of language is the arbitrariness of the relationship between sound and meaning. Sound symbolism, which refers to non-arbitrary mapping between the sound of a word and its meaning, contradicts this assumption. Sensitivity to sound symbolism has been studied through crossmodal correspondences (CCs) between auditory pseudowords (e.g. loh-moh) and visual shapes (e.g. a blob). We used representational similarity analysis to examine the relationships between physical stimulus parameters and perceptual ratings that varied on dimensions of roundedness and pointedness, for a range of auditory pseudowords and visual shapes. We found that perceptual ratings of these stimuli relate to certain physical features of both the visual and auditory domains. Representational dissimilarity matrices (RDMs) of parameters that capture the spatial profile of the visual shapes, such as the simple matching coefficient and Jaccard distance, were significantly correlated with those of the visual ratings. RDMs of certain acoustic parameters of the pseudowords, such as the temporal fast Fourier transform (FFT) and spectral tilt, that reflect spectral composition, as well as shimmer and speech envelope that reflect aspects of amplitude variation over time, were significantly correlated with those of the auditory perceptual ratings. RDMs of the temporal FFT (acoustic) and the simple matching coefficient (visual) were significantly correlated. These findings suggest that sound-symbolic CCs are related to basic properties of auditory and visual stimuli, and thus provide insights into the fundamental nature of sound symbolism and how this might evoke specific impressions of physical meaning in natural language.

neuroscience↗

bioRxiv2018

Neural Basis Of The Sound-Symbolic Crossmodal Correspondence Between Auditory Pseudowords And Visual Shapes

Sound symbolism refers to the association between the sounds of words and their meanings, often studied using the crossmodal correspondence between auditory pseudowords, e.g. takete or maluma, and pointed or rounded visual shapes, respectively. In a functional magnetic resonance imaging study, participants were presented with pseudoword-shape pairs that were sound-symbolically congruent or incongruent. We found no significant congruency effects in the blood oxygenation level-dependent (BOLD) signal when participants were attending to visual shapes. During attention to auditory pseudowords, however, we observed greater BOLD activity for incongruent compared to congruent audiovisual pairs bilaterally in the intraparietal sulcus and supramarginal gyrus, and in the left middle frontal gyrus. We compared this activity to independent functional contrasts designed to test competing explanations of sound symbolism, but found no evidence for mediation via language, and only limited evidence for accounts based on multisensory integration and a general magnitude system. Instead, we suggest that the observed incongruency effects are likely to reflect phonological processing and/or multisensory attention. These findings advance our understanding of sound-to-meaning mapping in the brain.

neuroscience↗