Biology subjects

Mickiewicz, E. A.

Publications and source records attributed to Mickiewicz, E. A..

3 recordsLinked to original sources

bioRxiv2025

Attention is all you need (in the brain): semantic contextualization in human hippocampus

Word meanings in language are contextualized by surrounding words. Inspired by the self-attention mechanism in transformer-based large language models (LLMs), we hypothesized that structural composition in the brain arises from combining canonical (non-contextual) word representations with those of nearby words. We analyzed single unit activity in the human hippocampus, a region involved in semantic and contextual processing, while n=10 participants listened to podcasts. We found that hippocampal neurons encoded word position within a clause, using both ordinal and frequency-domain positional encoding. Moreover, neural responses to specific words reflected both the words own lexical semantics and a weighted sum of the embeddings of preceding words. The relative weighting of these contextualizing words correlated with LLM self-attention weights. These findings suggest that contextualization in the brain makes use of vectorial shifts that have a resemblance to attentional reweighting in LLMs, and highlight the role of the mesial temporal lobe within the broader language network.

neuroscience↗

bioRxiv2025

Learning and language in the unconscious human hippocampus

Consciousness is a fundamental component of cognition,1 but the degree to which higher-order pattern recognition relies on it remains disputed.2,3 Here we demonstrate the persistence of oddball discrimination, semantic processing, and online prediction in individuals under general anesthesia-induced loss of consciousness.4,5 Using high-density Neuropixels microelectrodes6 to record both single unit and local field potential neural activity in the human hippocampus while playing a series of tones to anesthetized patients, we found that hippocampal neurons and local oscillations retained some detection of oddball tones. This effect size grew over the course of the experiment ([~]10 minutes), demonstrating representational plasticity. A biologically plausible recurrent neural network model showed that learning and oddball representation are an emergent property of flexible tone discrimination. Moreover, when we played language stimuli, single units and local field potentials carried information about the semantic and grammatical features of natural speech, even predicting semantic information about upcoming words. Together these results indicate that in the hippocampus, which is anatomically and functionally distant from primary sensory cortices,7 complex processing of sensory stimuli occurs even in the unconscious state.

neuroscience↗

bioRxiv2025

A vectorial code for semantics in human hippocampus

As we listen to speech, our brains track the meanings of the words we hear. Recent successes of large language models suggest that distributed population geometry can capture rich semantic relationships between words. Motivated by this idea, we hypothesized that semantic information in the brain may likewise be expressed in distributed patterns of activity across neurons, rather than in the activity of neurons narrowly tuned to a specific word. We recorded responses of hundreds of neurons in the human hippocampus while participants listened to narrative speech. We find encoding of contextual word meaning in the simultaneous activity of neurons whose individual selectivities span multiple unrelated semantic categories. Decoding and population geometry analyses revealed distinct neural coding principles for low-versus high-frequency words, likely reflecting the greater polysemy of common words. Similar to embedding vectors in semantic language models, distance between neural population responses correlates with semantic distance; however, this effect was only observed in contextual embedding models (GPT-2 and BERT), suggesting that the semantic distance effect depends critically on contextualization. Consistent with this, we find that neural population activity supports a multidimensional semantic subspace that aligns most closely with the contextual structure captured by GPT-2. Moreover, for semantically similar words, even contextual embedders showed an inverse correlation between semantic and neural distances; we attribute this pattern to the noise-mitigating benefits of contrastive coding. Ultimately, these results provide a neurocomputational account for understanding how neural populations track word meaning.

neuroscience↗