Thoughtful faces: inferring internal states across species using facial features

Tlaie, A.; Abd El Hay, M. Y.; Mert, B.; Taylor, R.; Ferracci, P. A.; Shapcott, K. A.; Glukhova, M.; Pillow, J. W.; Havenith, M. N.; Scholvinck, M.

doi:10.1101/2024.01.24.577055

bioRxiv · 10.1101/2024.01.24.577055

Thoughtful faces: inferring internal states across species using facial features

Tlaie, A.·Abd El Hay, M. Y.·Mert, B.·Taylor, R.·Ferracci, P. A.·Shapcott, K. A.·Glukhova, M.·Pillow, J. W.·Havenith, M. N.·Scholvinck, M.

View original source ↗Export metadata ↓

Abstract

Animal behaviour is shaped to a large degree by internal cognitive states, but it is unknown whether these states are similar across species. To address this question, here we develop a virtual reality setup in which male mice and macaques engage in the same naturalistic visual foraging task. We exploit the richness of a wide range of facial features extracted from video recordings during the task, to train a Markov-Switching Linear Regression (MSLR). By doing so, we identify, on a singletrial basis, a set of internal states that reliably predicts when the animals are going to react to the presented stimuli. Even though the model is trained purely on reaction times, it can also predict task outcome, supporting the behavioural relevance of the inferred states. The relationship of the identified states to task performance is comparable between mice and monkeys. Furthermore, each state corresponds to a characteristic pattern of facial features that partially overlaps between species, highlighting the importance of facial expressions as manifestations of internal cognitive states across species.

Source connections

Published article ↗

Explore related subjects

animal behavior and cognition Preprint

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Tlaie, A., Abd El Hay, M. Y., Mert, B., Taylor, R., Ferracci, P. A., Shapcott, K. A., Glukhova, M., Pillow, J. W., Havenith, M. N., Scholvinck, M.. 2024-01-29. Thoughtful faces: inferring internal states across species using facial features. https://doi.org/10.1101/2024.01.24.577055

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Shared brain transcriptome signatures of social evolution and plasticity in shell-dwelling cichlid fishes

Inter-individual variability in phenotypic traits is widespread in nature both across and within species, with similar patterns of variation often occurring between and within species. Given that these two types of variation are generated by evolutionary and plasticity processes respectively, it raises the question to what extent the causal mechanisms underlying such variability at these two different levels are shared or not. To address this question, we studied the variation in brain transcriptomes in Tanganyikan cichlids that exhibit both intra- and interspecific variation in social behavior. We compared two facultatively social cichlid species, each containing naturally occurring solitary and social-living individuals, with a closely related obligatory solitary resembling the ancestral state. Differential gene expression and Gene Set Enrichment Analyses were performed in the telencephalon and diencephalon transcriptomes to investigate the molecular basis of both behavioral plasticity and the evolution of sociality. Behavioral plasticity involved few differentially expressed genes but extensive changes in biological pathways, supporting a highly polygenic architecture of social behavior. Across both species, social-living individuals consistently upregulated pathways involved in translation, protein homeostasis, and energy metabolism, whereas solitary individuals showed enrichment of neuronal morphogenesis, cell adhesion, and structural plasticity. Remarkably, these same functional programs were associated with the independent evolution of sociality, revealing a strong overlap between evolutionary and plastic transcriptional responses. This convergence was largely restricted to the telencephalon, while the diencephalon showed limited and inconsistent functional signatures. Together, our results support the repeated recruitment of gene regulatory programs during the evolution of sociality and demonstrate that behavioral plasticity and evolutionary transitions share common neurogenomic responses, consistent with a conserved genetic toolkit for social behavior.

animal behavior and cognition↗

bioRxiv2026

Causal Multiscale Modeling of Ocular FLASH Electroretinography: Model Development, Experimental Fitting, and Spin-Glass--Percolation Predictions

\textbf{Purpose:} To develop a causal multiscale model of retinal electroretinography (ERG) after ocular ultrahigh-dose-rate (FLASH) and conventional-dose-rate (CONV) irradiation, evaluate the framework against published ocular ERG data, and derive prospective dose-response predictions for future validation. \textbf{Methods:} The retina was coarse-grained into $N=200$ functional modules. Local rate equations were formulated as Hodgkin--Huxley-inspired conductance-balance/RC reductions of retinal ion-channel and synaptic pathways. Radiation generated an injury field that altered phototransduction, a-wave and b-wave dynamics and, in the network extension, inter-module coupling and bond survival. A reduced amplitude benchmark and nested M0/M1/M2 causal models were evaluated. \textbf{Results:} The reduced benchmark achieved $R^2=0.9721$ and RMSE $=11.94~\mu$V. M0, M1, and M2 yielded $R^2=0.8546$, 0.8559, and 0.8560, respectively, indicating limited independent identifiability of the network terms from amplitude data alone. Under an exploratory 50\% retained-gain criterion, the reduced benchmark predicted a-wave preservation thresholds of 28.6 Gy for FLASH and 23.7 Gy for CONV, and b-wave thresholds of 29.2 Gy and 24.1 Gy, respectively, corresponding to a model-based functional dose-modifying factor (DMF) of 1.21. \textbf{Conclusion:} The ERG amplitude data support the causal radiation-injury-to-ERG framework, while additional observables will be needed to distinguish the predicted collective-network contributions. The model also provides quantitative dose targets for prospective studies of the FLASH retinal-preservation window and a framework for testing how local retinal injury may propagate into collective dysfunction.

animal behavior and cognition↗

bioRxiv2026

Neonatal personality is associated with sex-specific developmental trajectories in a wild mammal

Animal personality, defined as consistent among-individual differences in behaviour, is an important source of phenotypic variation, but its developmental origins and associations with subsequent development remain poorly understood. We measured neonatal personalities, early-life growth, and yearling male antler length in wild fallow deer (Dama dama). Using eight years of data, we estimated repeatability and among-individual correlations among neonatal personalities and growth and applied structural equation modelling to examine associations with yearling male morphology. Heart rate and behavioural reaction measures showed repeatable among-individual variation, demonstrating that stable differences were detectable shortly after birth. Behavioural reaction was negatively correlated with growth in males but not in females. Heart rate was not associated with growth in either sex. In males, the effects of neonatal behavioural reactivity persisted beyond early life: more behaviourally reactive neonates had shorter antlers as yearlings, indirectly through reduced early-life growth. These findings show that stable among-individual differences emerge during the neonatal period and are associated with sex-specific developmental trajectories.

animal behavior and cognition↗