Biology subjects

Fiebach, C. J.

Publications and source records attributed to Fiebach, C. J..

10 recordsLinked to original sources

bioRxiv2024

Beyond Letters: Optimal Transport as a Model for Sub-Letter Orthographic Processing

Letter processing plays a key role in visual word recognition. However, word recognition models typically overlook or greatly simplify early perceptual processes of letter recognition. We suggest that optimal transport theory may provide a computational framework for describing letter shape processing. We use representational similarity analysis to show that optimal transport cost (Wasserstein distance) between pairs of letters aligns with neural activity elicited by visually presented letters <225 ms after stimulus onset, outperforming an existing approach based on shape overlap. We additionally show that optimal transport can capture the emergence of geometric invariances (e.g., to position or size) observed in letter perception. Finally, we demonstrate that Wasserstein distance predicts neural activity similarly well to features from artificial networks trained to classify images and letters. However, whereas representations in artificial neural networks emerge in a computationally unconstrained manner, our proposal provides a computationally explicit route to modeling the earliest orthographic processes.

neuroscience↗

bioRxiv2024

Non-Human Recognition of Orthography: How is it implemented and how does it differ from Human orthographic processing

The ability to robustly recognize strings of letters, a cornerstone of reading, was observed in baboons and pigeons despite their lack of phonological and semantic knowledge. Here, we apply a comparative modeling approach to investigate the neuro-cognitive basis of orthographic decision behavior in humans, baboons, and pigeons, addressing whether phylogenetic relatedness entails similar underlying neuro-cognitive phenotypes. We use the highly transparent SpeechLess Reader (SLR) model, which assumes letter-string recognition based on a computational implementation of predictive coding, whereby orthographic decisions rely on prediction-error signals emerging from multiple representational levels: visual-pixel, letter, and letter-sequence representations. We investigate which representations species use during successful orthographic decision-making. We introduce multiple SLR variants, each including one or more prediction-error representations, and compare the simulations of each variant with orthographic decisions from individuals of three species after learning letter strings without meaning. Humans predominantly relied on letter-sequence-level representations, resulting in the highest task performance in behavior and model simulations. Baboons also relied on sequence-based representations, but in combination with pixel- and letter-level representations. In contrast, pigeons relied more on pixel and letter-level representations. These findings suggest that the orthographic representations used in orthographic decisions reflect phylogenetic distance: Humans and baboons use more similar representations than pigeons. Overall, the description of orthographic decisions based on a small set of representations and computations was highly successful in describing behavior, even for humans who mastered reading in its entirety.

animal behavior and cognition↗

bioRxiv2020

Predictive pre-activation of orthographic and lexical-semantic representations facilitates visual word recognition

To a crucial extent, the efficiency of reading results from the fact that visual word recognition is faster in predictive contexts. Predictive coding models suggest that this facilitation results from pre-activation of predictable stimulus features across multiple representational levels before stimulus onset. Still, it is not sufficiently understood which aspects of the rich set of linguistic representations that are activated during reading - visual, orthographic, phonological, and/or lexical-semantic - contribute to context-dependent facilitation. To investigate in detail which linguistic representations are pre-activated in a predictive context and how they affect subsequent stimulus processing, we combined a well-controlled repetition priming paradigm, including words and pseudowords (i.e., pronounceable nonwords), with behavioral and magnetoencephalography measurements. For statistical analysis, we used linear mixed modeling, which we found had a higher statistical power compared to conventional multivariate pattern decoding analysis. Behavioral data from 49 participants indicate that word predictability (i.e., context present vs. absent) facilitated orthographic and lexical-semantic, but not visual or phonological processes. Magnetoencephalography data from 38 participants show sustained activation of orthographic and lexical-semantic representations in the interval before processing the predicted stimulus, suggesting selective pre-activation at multiple levels of linguistic representation as proposed by predictive coding. However, we found more robust lexical-semantic representations when processing predictable in contrast to unpredictable letter strings, and pre-activation effects mainly resembled brain responses elicited when processing the expected letter string. This finding suggests that pre-activation did not result in explaining away predictable stimulus features, but rather in a sharpening of brain responses involved in word processing.

neuroscience↗

bioRxiv2019

Traces of Meaning Itself: Encoding distributional word vectors in brain activity

How is semantic information stored in the human mind and brain? Some philosophers and cognitive scientists argue for vectorial representations of concepts, where the meaning of a word is represented as its position in a high-dimensional neural state space. At the intersection of natural language processing and artificial intelligence, a class of very successful distributional word vector models has developed that can account for classic EEG findings of language, i.e., the ease vs. difficulty of integrating a word with its sentence context. However, models of semantics have to account not only for context-based word processing, but should also describe how word meaning is represented. Here, we investigate whether distributional vector representations of word meaning can model brain activity induced by words presented without context. Using EEG activity (event-related brain potentials) collected while participants in two experiments (English, German) read isolated words, we encode and decode word vectors taken from the family of prediction-based word2vec algorithms. We find that, first, the position of a word in vector space allows the prediction of the pattern of corresponding neural activity over time, in particular during a time window of 300 to 500 ms after word onset. Second, distributional models perform better than a human-created taxonomic baseline model (WordNet), and this holds for several distinct vector-based models. Third, multiple latent semantic dimensions of word meaning can be decoded from brain activity. Combined, these results suggest that empiricist, prediction-based vectorial representations of meaning are a viable candidate for the representational architecture of human semantic knowledge.

neuroscience↗

bioRxiv2018

ADHD Symptoms are Associated with the Modular Structure of Intrinsic Brain Networks in a Representative Sample of Healthy Adults

Attention-deficit/hyperactivity disorder (ADHD) is one of the most common neurodevelopmental disorders with significant and often lifelong effects on social, emotional, and cognitive functioning. Influential neurocognitive models of ADHD link behavioral symptoms to altered connections between and within functional brain networks. Here, we investigate whether network-based theories of ADHD can be generalized to understanding variations in ADHD-related behaviors within the normal (i.e., clinically unaffected) adult population. In a large and representative sample, self-rated presence of ADHD symptoms varied widely; only eight out of 291 participants scored in the clinical range. Subject-specific brain-network graphs were modeled from functional MRI resting-state data and revealed significant associations between (non-clinical) ADHD symptoms and region-specific profiles of between-module and within-module connectivity. Effects were located in brain regions associated with multiple neuronal systems including the default-mode network, the salience network, and the central executive system. Our results are consistent with network perspectives of ADHD and provide further evidence for the relevance of an appropriate information transfer between task-negative (default-mode) and task-positive brain regions. More generally, our findings support a dimensional conceptualization of ADHD and contribute to a growing understanding of cognition as an emerging property of functional brain networks. Author SummaryNeurocognitive models of ADHD link behavioral symptoms to altered connections between and within functional brain networks. We investigate whether these network-based theories of ADHD can be generalized to ADHD-related behaviors within the normal adult population. Subject-specific brain graphs were modeled from functional MRI resting-state data of a large and representative sample (N = 291). Significant associations between ADHD-related behaviors and region-specific profiles of between-module and within-module connectivity were observed in brain regions associated with multiple functional systems including the default-mode network, the salience network, and the central executive system. Our results support a dimensional conceptualization of ADHD and enforce network-based models of ADHD by providing further evidence for the relevance of an appropriate information transfer between task-negative (default-mode) and task-positive brain regions.

neuroscience↗

bioRxiv2018

Visual word recognition relies on an orthographic prediction error signal

Most current models assume that the perceptual and cognitive processes of visual word recognition and reading operate upon neuronally coded domain-general low-level visual representations - typically oriented line representations. We here demonstrate, consistent with neurophysiological theories of Bayesian-like predictive neural computations, that prior visual knowledge of words may be utilized to explain away redundant and highly expected parts of the visual percept. Subsequent processing stages, accordingly, operate upon an optimized representation of the visual input, the orthographic prediction error, highlighting only the visual information relevant for word identification. We show that this optimized representation is related to orthographic word characteristics, accounts for word recognition behavior, and is processed early in the visual processing stream, i.e., in V4 and before 200 ms after word-onset. Based on these findings, we propose that prior visual-orthographic knowledge is used to optimize the representation of visually presented words, which in turn allows for highly efficient reading processes.

neuroscience↗

bioRxiv2018

Dissociable prelexical and lexical contributions to visual word recognition and priming: Evidence from MEG and behavior

Word familiarity and predictive context facilitate visual word processing, leading to faster recognition times and reduced neuronal responses. Previously, models with and without top-down connections, including lexical-semantic, pre-lexical (e.g., orthographic/ phonological), and visual processing levels were successful in accounting for these facilitation effects. Here we systematically assessed context-based facilitation with a repetition priming task and explicitly dissociated pre-lexical and lexical processing levels using a pseudoword familiarization procedure. Experiment 1 investigated the temporal dynamics of neuronal facilitation effects with magnetoencephalography (MEG; N=38 human participants) while Experiment 2 assessed behavioral facilitation effects (N=24 human participants). Across all stimulus conditions, MEG demonstrated context-based facilitation across multiple time windows starting at 100 ms, in occipital brain areas. This finding indicates context based-facilitation at an early visual processing level. In both experiments, we furthermore found an interaction of context and lexical familiarity, such that stimuli with associated meaning showed the strongest context-dependent facilitation in brain activation and behavior. Using MEG, this facilitation effect could be localized to the left anterior temporal lobe at around 400 ms, indicating within-level (i.e., exclusively lexical-semantic) facilitation but no top-down effects on earlier processing stages. Increased pre-lexical familiarity (in pseudowords familiarized utilizing training) did not enhance or reduce context effects significantly. We conclude that context based-facilitation is achieved within visual and lexical processing levels. Finally, by testing alternative hypotheses derived from mechanistic accounts of repetition suppression, we suggest that the facilitatory context effects found here are implemented using a predictive coding mechanism. Significance StatementThe goal of reading is to derive meaning from script. This highly automatized process benefits from facilitation depending on word familiarity and text context. Facilitation might occur exclusively within each level of word processing (i.e., visual, pre-lexical, and/or lexical-semantic) but could alternatively also propagate in a top-down manner from higher to lower levels. To test the relevance of these two alternative accounts at each processing level, we combined a pseudoword learning approach controlling for letter string familiarity with repetition priming. We found enhanced context-based facilitation at the lexical-semantic but not pre-lexical processing stage, and no evidence of top-down facilitation from lexical-semantic to earlier word recognition processes. We also identified predictive coding as the most likely mechanism underlying within-level context-based facilitation.

neuroscience↗

bioRxiv2018

Decoding semantic predictions from EEG prior to word onset

ABSTRACTThe outstanding speed of language comprehension necessitates a highly efficient implementation of cognitive-linguistic processes. The domain-general theory of Predictive Coding suggests that our brain solves this problem by continuously forming linguistic predictions about expected upcoming input. The neurophysiological implementation of these predictive linguistic processes, however, is not yet understood. Here, we use EEG (human participants, both sexes) to investigate the existence and nature of online-generated, category-level semantic representations during sentence processing. We conducted two experiments in which some nouns - embedded in a predictive spoken sentence context - were unexpectedly delayed by 1 second. Target nouns were either abstract/concrete (Experiment 1) or animate/inanimate (Experiment 2). We hypothesized that if neural prediction error signals following (temporary) omissions carry specific information about the stimulus, the semantic category of the upcoming target word is encoded in brain activity prior to its presentation. Using time-generalized multivariate pattern analysis, we demonstrate significant decoding of word category from silent periods directly preceding the target word, in both experiments. This provides direct evidence for predictive coding during sentence processing, i.e., that information about a word can be encoded in brain activity before it is perceived. While the same semantic contrast could also be decoded from EEG activity elicited by isolated words (Experiment 1), the identified neural patterns did not generalize to pre-stimulus delay period activity in sentences. Our results not only indicate that the brain processes language predictively, but also demonstrate the nature and sentence-specificity of category-level semantic predictions preactivated during sentence comprehension.\n\nSTATEMENT OF SIGNIFICANCEThe speed of language comprehension necessitates a highly efficient implementation of cognitive-linguistic processes. Predictive processing has been suggested as a solution to this problem, but the underlying neural mechanisms and linguistic content of such predictions are only poorly understood. Inspired by Predictive Coding theory, we investigate whether the meaning of expected, but not-yet heard words can be decoded from brain activity. Using EEG, we can predict if a word is, e.g., abstract (as opposed to concrete), or animate (vs. inanimate), from brain signals preceding the word itself. This strengthens predictive coding theory as a likely candidate for the principled neural mechanisms underlying online processing of language and indicates that predictive processing applies to highly abstract categories like semantics.

neuroscience↗

bioRxiv2018

Reading at the speed of speech: the rate of eye movements aligns with auditory language processing

Across languages, the speech signal is characterized by a predominant modulation of the amplitude spectrum between about 4.3-5.5Hz, reflecting the production and processing of linguistic information chunks (syllables, words) every [~]200ms. Interestingly, [~]200ms is also the typical duration of eye fixations during reading. Prompted by this observation, we demonstrate that German readers sample written text at [~]5Hz. A subsequent meta-analysis with 142 studies from 14 languages replicates this result, but also shows that sampling frequencies vary across languages between 3.9Hz and 5.2Hz, and that this variation systematically depends on the complexity of the writing systems (character-based vs. alphabetic systems, orthographic transparency). Finally, we demonstrate empirically a positive correlation between speech spectrum and eye-movement sampling in low-skilled readers. Based on this convergent evidence, we propose that during reading, our brains linguistic processing systems imprint a preferred processing rate, i.e., the rate of spoken language production and perception, onto the oculomotor system.

neuroscience↗

bioRxiv2016

The lexical categorization model: A computational model of left ventral occipito-temporal cortex activation in visual word recognition

To characterize the left-ventral occipito-temporal cortex (lvOT) role during reading in a quantitatively explicit and testable manner, we propose the lexical categorization model (LCM). The LCM assumes that lvOT optimizes linguistic processing by allowing fast meaning access when words are familiar and filter out orthographic strings without meaning. The LCM successfully simulates benchmark results from functional brain imaging. Empirically, using functional magnetic resonance imaging, we demonstrate that quantitative LCM simulations predict lvOT activation across three studies better than alternative models. Besides, we found that word-likeness, which is assumed as input to LCM, is represented posterior to lvOT. In contrast, a dichotomous word/non-word contrast, which is assumed as the LCMs output, could be localized to upstream frontal brain regions. Finally, we found that training lexical categorization results in more efficient reading. Thus, we propose a ventral-visual-stream processing framework for reading involving word-likeness extraction followed by lexical categorization, before meaning extraction.

neuroscience↗