Search bioRxiv⌕ Search

Biology subjects

Schmaelzle, R.

Publications and source records attributed to Schmaelzle, R..

10 recordsLinked to original sources

Modeling Synthetic Audience Reactions to Social-Cognitive Narratives: A Generative GSR Model to Predict Real-World Autonomic Alignment during Film Viewing

While media consumption can be a solitary act, it produces a shared, socially coordinated experience where audiences bodies align in response to shared narrative events that are often social-affective in nature. Despite this recognition, traditional descriptive models of Galvanic skin response (GSR) have existed for decades, yet the socially coordinated aspect remains to be fully reflected in physiological models with the field of communication often treating the underlying generators of autonomic activity as a black box. To bridge this gap, we introduce a computational framework that models the underlying neural driver and its convolution to sweat gland physiology to explain how narrative events translate into measurable conductance. By leveraging multimodal AI models to "interpret" the social-cognitive content of a film, we generated a predictor timeline for a synthetic audience comprised of digital agents (i.e. artificial body systems responding to the film events with GSR responses). We then test this computational audience model by comparing its predictions against an empirical dataset collected as audience members (N = 96) processed the same stimulus, finding that AI-identified social triggers, like moments of comedic violence or shared emotional shifts, significantly predict the GSR time-course of audience engagement. In sum, this paper moves beyond simple and often retrospective labels like "arousal" to offer a computational account of how shared social narratives grip the human nervous system. We provide a scalable and expandable framework and a set of tools to predict media impact and understanding the psychophysiological basis of media.

animal behavior and cognition↗

The Signal in the Noise: Hierarchy and Robustness of Physiological Audience Alignment during Narrative Media

Communication research traditionally prioritizes receiver variance, often overlooking how a single message induces commonality across distinct systems. Drawing on classical theory, we propose that effective messages entrain the biological rhythms of the audience. In a large-scale experiment (N=198) monitoring eye-tracking, heart rate, and electrodermal activity during a film, we mapped this alignment. Results reveal a processing hierarchy: alignment was strongest for information acquisition (gaze) but also extended to downstream autonomic regulation. Crucially, a temporal manipulation confirmed this state is content-locked: alignment vanished when the narrative sequence was altered but was restored upon computational unscrambling of the physiological time series. This confirms that responses carry a temporal fingerprint of the content. We conclude that messages function as alignment devices, reducing individual noise to create a network of receivers exhibiting shared processing states, thus providing a materially grounded, signal-based definition of the audience.

animal behavior and cognition↗

Speaker responses to audience-induced social-evaluative threat: Evidence from scientific presentation tasks in immersive virtual reality

Success in public speaking hinges on engaging an audience - a high-stakes social interaction that remains a significant source of anxiety and stress for many. Using a virtual-reality (VR) paradigm, we tested how speakers delivering scientific talks perceive and respond to supportive vs. unsupportive audiences. We collected behavioral (eye contact, speech rate, motion expressiveness/openness), physiological (heart rate, EEG, breathing rate, pupil dilation), and self-report measures to assess audience effects. The unsupportive audience elicited greater negative affect, arousal, and anxiety, and higher perceived cognitive and social effort. Physiologically and behaviorally, speaking to the unsupportive audience slowed the speaking rate, and acoustic analyses further indicated greater emotional arousal and vocal dominance in the unsupportive condition. Finally, VR exposure reduced speaking anxiety overall. These findings highlight VR combined with physiological measurement as a powerful approach for investigating audience effects and social-communication processes, with clear implications for augmenting social intelligence and communication skills.

scientific communication and education↗

Memory Inception through Gaze-Contingent Message Exposure: Using Virtual Reality to Study Media Influence

The messages we encounter in our environment can shape our knowledge about the world. However, much research on mediadriven influence via messages focuses on population-level effects and aggregate exposure statistics, obscuring how individual and self-determined behaviors affect message intake, processing, and effects. To address this gap, we use virtual reality (VR) to create a controlled messaging environment. Participants navigate a simulated urban street lined with billboard messages while their visual attention is tracked via eye-tracking. We introduce an inception-style manipulation: overlooked billboards are strategically reintroduced, creating additional exposure opportunities. Our results demonstrate that this subtle manipulation - unnoticed by participants - boosts message retention. This study bridges communication theory and psychology, elucidating the blurred line between voluntary and involuntary information intake in the digital age. It also highlights a potential vulnerability in the future metaverse media ecosystem, where undetected information manipulations can influence individual and collective attention and memories.

animal behavior and cognition↗

Charting the Silent Signals of Social Gaze: Automating Eye Contact Assessment in Face-to-Face Conversations

Social gaze is a crucial yet often overlooked aspect of nonverbal communication. During conversations, it typically operates subconsciously, following automatic co-regulation patterns. However, deviations from typical patterns, such as avoiding eye contact or excessive gazing, can significantly affect social interactions and perceived relationship quality. The principles and effects of social gaze have intrigued researchers across various fields, including communication science, social psychology, animal biology, and psychiatry. Despite its significance, research in social gaze has been limited by methodological challenges in assessing eye movements and gaze direction during natural social interactions. To address these obstacles, we have developed a new approach combining mobile eye tracking technology with automated analysis tools. In this paper, we introduce, validate, and apply a pipeline for recording and analyzing gaze behavior in dyadic conversations. We present a sample study where dyads engaged in two types of interactions: a get-to-know conversation and a conflictual conversation. Our new analysis pipeline corroborated previous findings, such as people directing more eye gaze while listening than talking, and gaze typically lasting about three seconds before averting. These results demonstrate the potential of our methodology to advance the study of social gaze in natural interactions.

animal behavior and cognition↗

Using VR and eye-tracking to study attention to and retention of AI-generated ads in outdoor advertising environments

In contemporary urban environments, advertisements are ubiquitous, capturing the attention of individuals navigating through cityscapes. This study simulates this situation by using VR as an advertising research tool and combining it with eye-tracking to rigorously assess attention to and retention of visual advertisements. Specifically, participants drove through a virtual city with 40 AI-generated, experimentally manipulated, and randomly assigned advertisements (20 commercial, 20 health) distributed throughout. Our results confirm theoretical predictions about the link between exposure, visual attention, and incidental memory. Specifically, we found that attended ads are likely to be recalled and recognized, and concurrent task demands (counting sales signs) decreased visual attention and subsequent recall and recognition of the ads. Finally, we identify the influence of ad placement in the city as a very important but previously hard-to-study factor influencing advertising effects. This paradigm offers great flexibility for biometric advertising research and can be adapted to varying contexts, including highways, airports, and subway stations, and theoretically manipulate other variables. Moreover, considering the metaverse as a next-generation advertising environment, our work demonstrates how causal mechanisms can be identified in ways that are of equally high interest to both theoretical as well as applied advertising research.

animal behavior and cognition↗

The eyes have it: Inter-subject correlations of pupillary responses for audience response measurement in VR

The eye is the vanguard of the reception process, constituting the point where visual information arrives and is transformed into neural signals. While we view dynamic media contents, a fine-tuned interplay of mechanisms causes our pupils to dilate and constrict over time - and putatively similarly across audience members exposed to the same messages. Research that once pioneered pupillometry did actually use dynamic media as stimuli, but this trend then stalled, and pupillometry remained underdeveloped in the study of naturalistic media stimuli. Here, we introduce a VR-based approach to capture audience members pupillary responses during media consumption and suggest an innovative analytic framework. Specifically, we expose audiences to a set of 30 different video messages and compute the cross-receiver similarity of pupillometric responses. Based on this data, we identify the specific video an individual is watching. Our results show that this pupil-pulse-tracking enables highly accurate decoding of video identity. Moreover, we demonstrate that the decoding is relatively robust to manipulations of video size and distractor presence. Finally, we examine the relationship between pupillary responses and subsequent memory. Theoretical implications for objectively quantifying exposure and states of audience engagement are discussed. Practically, we anticipate that this pupillary audience response measurement approach could find application in media measurement across contexts, ranging from traditional screen-based media (commercials, movies) to social media (e.g., TikTok and YouTube), and to next-generation virtual media environments (e.g., Metaverse, gaming).

physiology↗

The VR Billboard Paradigm: Using VR and Eye-tracking to Examine the Exposure-Reception-Retention Link in Realistic Communication Environments

Exposure is the cornerstone of media and message effects research. If a health, political, or commercial message is not noticed, no effects can ensue. Yet, existing research in communication, advertising, and related disciplines often fails to measure exposure and demonstrate the causal link between quantified exposure to outcomes because actual exposure (i.e., whether recipients were not only exposed to messages but also took notice of them) is difficult to capture. Here, we harness Virtual Reality (VR) technology integrated with eye tracking to overcome this challenge. While eye-tracking technology alone can capture whether people attend to messages in their communication environment, most eye-tracking research is bound by laboratory-based screen-reading paradigms that are not representative of the broader communication environments in which messages are encountered. Emerging eye-tracking field research suffers from an inability to control and experimentally manipulate key variables. Our solution is to measure eye-tracking within an immersive environment in VR that resembles a realistic message reception context. Specifically, we simulate driving down a highway alongside which billboards are placed and use VR-integrated eye-tracking to measure whether the drivers look at individual billboard messages. This allows us to rigorously quantify the nexus between exposure and reception, and to link our measures to subsequent memory, i.e., whether messages were remembered, forgotten, or not even encoded. We further demonstrate that manipulating drivers attention directly impacts gaze behavior and memory. We discuss the large potential of this paradigm to quantify exposure and message reception in realistic communication environments and the equally promising applications in new media contexts (e.g., the Metaverse).

animal behavior and cognition↗

Reliability of fMRI time series:Similarity of neural processing during movie viewing

Despite its widespread use in neuroscience, the reliability of fMRI remains insufficiently understood. One powerful way to tap into aspects of fMRI reliability is via the inter-subject correlation (ISC) approach, which exposes different viewers to the same time-locked naturalistic stimulus and assesses the similarity of neural time series. Here we examined the correlations of fMRI time series from 24 participants who watched the same movie clips across three repetitions. This enabled us to examine inter-subject correlations, intra-subject correlations, and correlations between aggregated time series, which we link to the notions of inter-rater reliability, stability, and consistency. In primary visual cortex we found average pairwise inter-subject correlations of about r = 0.3, and intra-subject correlations of similar magnitude. Aggregation across subjects increased inter-subject (inter-group) correlations to r = 0.87, and additional intra-subject averaging before cross-subject aggregation yielded correlations of r = 0.93. Computing the same analyses for parietal (visuospatial network) and cingulate cortices (saliency network) revealed a gradient of decreasing ISC from primary visual to higher visual to post-perceptual regions. These latter regions also benefitted most from the increased reliability due to aggregation. We discuss theoretical and practical implications of this link between neural process similarity and psychometric conceptions of inter-rater reliability, stability, and internal consistency.

neuroscience↗

Brain connectivity dynamics during social interaction reflect social network structure

Social ties are crucial for humans. Disruption of ties through social exclusion has a marked effect on our thoughts and feelings; however, such effects can be tempered by broader social network resources. Here, we use functional magnetic resonance imaging data acquired from 80 male adolescents to investigate how social exclusion modulates functional connectivity within and across brain networks involved in social pain and understanding the mental states of others (i.e., mentalizing). Furthermore, using objectively logged friendship network data, we examine how individual variability in brain reactivity to social exclusion relates to the density of participants friendship networks, an important aspect of social network structure. We find increased connectivity within a set of regions previously identified as a mentalizing system during exclusion relative to inclusion. These results are consistent across the regions of interest as well as a whole-brain analysis. Next, examining how social network characteristics are associated with task-based connectivity dynamics, participants who showed greater changes in connectivity within the mentalizing system when socially excluded by peers had less dense friendship networks. This work provides novel insight to understand how distributed brain systems respond to social and emotional challenges, and how such brain dynamics might vary based on broader social network characteristics.

neuroscience↗