Mezey, D. | Search bioRxiv

bioRxiv2025

Real-Time Human Interaction with Virtual Swarms in Shared Physical Space

Human-swarm interaction (HSI) explores how humans engage with distributed collective systems, aiming to incorporate human cognition into scalable and robust robotic swarms. While most HSI research focuses on remote teleoperation via engineered interfaces, real-world integration of swarms into everyday tasks requires natural, embodied interactions in shared physical spaces. To address the limitations of traditional teleoperation studies, and the high resource demands of using physical robot swarms for HSI research, we introduce CoBe XR, a spatial augmented reality system that projects virtual swarms into the physical environment of the human operator. CoBe XR enables real-time, fine-grained, natural interaction between humans and swarm-like agents through full-body movement without dedicated control interfaces or prior training. As a proof-of-concept, we present a behavioral study involving 40 participants who influenced swarm behavior solely through walking. Our results show that human participants were able to adapt to the collective dynamics of the swarm and control it through natural perception-motion control in a shared physical space. We argue that similar extended reality systems can not only reveal how humans perceive and adapt to collective dynamics, but they offer a general platform to understand human behavior or an intermediate solution to design embodied robot swarms.

animal behavior and cognition↗

bioRxiv2023

Visual social information use in collective foraging

Collective dynamics emerge from individual-level decisions, yet we still poorly understand the link between individual-level decision-making processes and collective outcomes in realistic physical systems. Using collective foraging to study the key trade-off between personal and social information use, we present a mechanistic, spatially-explicit agent-based model that combines individual-level evidence accumulation of personal and (visual) social cues with particle-based movement. Under idealized conditions without physical constraints, our mechanistic framework reproduces findings from established probabilistic models, but explains how individual-level decision processes generate collective outcomes in a bottom-up way. In clustered environments, groups performed best if agents reacted strongly to social information, while in uniform environments, individualistic search was most beneficial. Incorporating different real-world physical and perceptual constraints profoundly shaped collective performance, and could even buffer maladaptive herding by facilitating self-organized exploration. Our study uncovers the mechanisms linking individual cognition to collective outcomes in human and animal foraging and paves the way for decentralized robotic applications. Significance statementFinding and collecting rewards in heterogeneous environments is key for adaptive collective behavior in humans, animals and machines. We present an open agent-based simulation framework to study how social information use shapes collective foraging from the bottom up. Our model combines individual evidence accumulation with spatially explicit movement. Our results connect individual-level decisions to collective dynamics in realistic physical environments, highlighting the key role of real-world constraints, thereby bringing us closer to embodied collective intelligence. Our work introduces a flexible platform to study the interplay between individual cognitive and perceptual biases, agents physical environment and the resulting collective dynamics and thus paves the way for fully decentralized mobile robot applications.

animal behavior and cognition↗