Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.64898/2026.10.07.757484

JEPA-SORT: World-Model Embeddings Improve Deep-Sea Multi-Object Tracking on Visually Homogeneous Fauna

Tillinghast-Raby, A.·Kravetz, M.·Kamau-Weng, J.·Weinnig, A.·Rodrigues, Y.·Varini, F.·Pieribone, V.
View original source ↗Export metadata ↓

Abstract

Protecting deep-sea sessile communities as Marine Protected Areas (MPAs) requires accurate estimates of their spatial distribution and abundance. Existing computer-vision detection methods require careful calibration to avoid counting errors; multi-object tracking (MOT) theoretically offers a more robust alternative. In practice, however, poor lighting and turbidity limit the efficacy of purely kinematic MOT algorithms in deep-sea environments. At the same time, the intra-species visual homogeneity typical of deep-sea scenes means traditional per-frame appearance features are often too weak to maintain stable identities in deep-association MOT algorithms. In this paper we introduce JEPA-SORT, an MOT algorithm that injects V-JEPA 2.1 embeddings into detection-to-track association, to show that video world-model embeddings - which capture not only physical appearance but spatiotemporal trajectory across viewpoints - can help maintain stable identities on visually homogeneous specimens. We provide a benchmark of twelve ROV transects surveying deep-sea sessile communities and evaluate JEPA-SORT against standard MOT algorithms across a range of simulated optical environments. We also propose the Biodensity Utility Metric (BDU), which measures tracker performance in the context of downstream ecological modeling. JEPA-SORT outperforms all benchmarked trackers on IDF1, MOTA, and BDU and is competitive on specimen-count accuracy. Ablations indicate its advantage stems primarily from the discriminative power of world-model embeddings on visually homogeneous footage.

Explore related subjects

ecologyPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Tillinghast-Raby, A., Kravetz, M., Kamau-Weng, J., Weinnig, A., Rodrigues, Y., Varini, F., Pieribone, V.. 2026-10-09. JEPA-SORT: World-Model Embeddings Improve Deep-Sea Multi-Object Tracking on Visually Homogeneous Fauna. https://doi.org/10.64898/2026.10.07.757484

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Climate change and extreme weather events to intensify West Nile virus activity

Vector-borne zoonotic diseases pose a substantive risk to public health worldwide, yet predicting outbreaks at scales useful for public health response remains a critical challenge. Transmitted by Culex mosquitoes, West Nile virus is the leading cause of arboviral disease in the United States, progressing to neuroinvasive disease in approximately 1% of infections. Arizona's record-breaking 2021 outbreak, with 1,715 reported human cases and 127 deaths exemplifies this growing burden. Using a decade of mosquito surveillance data coupled with time-lagged weather patterns in a random forest framework, we developed spatial models predicting Culex abundance and West Nile virus occurrence across Arizona at weekly temporal and fine spatial resolution. Our weighted models classified viral occurrence with 98.9% accuracy and explained 65.3% of weekly variation in county-level human cases, with human cases lagging modeled activity by one week. Neither mosquito abundance nor viral occurrence declined at the extreme temperatures characteristic of Arizona summers, suggesting that thermal refugia sustain transmission. Drought-monsoon cycling, rather than temperature alone, emerged as the primary driver of West Nile activity, and projections through 2100 indicate intensifying activity under all climate change scenarios.

ecology↗
bioRxiv2026

Differing patterns in community composition and diversity of biofilm-forming bacteria and fungi along a large salinity gradient

Aim: Biofilms are complex ecosystems formed by phototrophic and heterotrophic organisms, such as bacteria and fungi. Biofilms are hotspots for microbial activity and crucial for the functioning of all aquatic ecosystems, but they are overlooked and poorly studied, especially along large environmental gradients. We aimed at resolving the responses of bacterial and fungal communities to environmental variation along a gradient with strong variation in environmental factors, especially salinity. We also investigated whether the diversity of biofilm-forming bacteria and fungi follows the pattern predicted by the Remane curve, which posits that species diversity is lowest at intermediate salinity. Location: 2300-km-long spatial gradient in the Baltic Sea, Northern Europe Time period: August 2018 Major taxa studied: Biofilm-forming bacteria and fungi Methods: We investigated the composition and diversity of bacteria and fungi by collecting biofilm samples from 42 shallow benthic ecosystems. We also characterized key environmental variables, such as salinity, nutrient concentrations, wind exposure, and climatic conditions. Bacterial and fungal communities in biofilm samples were identified using Illumina MiSeq sequencing. Results: We found that bacteria and fungi responded differently to environmental variation. Bacterial communities were mainly controlled by salinity, and their diversity was lowest in the intermediate salinity. Bacterial communities were also dominated by different bacterial orders in high, intermediate, and low salinity. Fungal communities, instead, were jointly controlled by several environmental variables, i.e., salinity, water temperature, wind exposure, and nutrients, and their diversity or composition did not show a clear pattern along the salinity gradient. Main conclusions: To our knowledge, this is one of the first studies to investigate the biofilm-forming communities, especially fungal communities, along large environmental gradients. Our results highlight the role of biofilms as complex ecosystems where organisms ecological requirements are unique and variable.

ecology↗
bioRxiv2026

Reinforcement Learning Reveals Spatial Strategies for Slowing Biological Invasions

Managing biological invasions requires deciding where, when, and how to act while populations continue to grow and spread. Invasive species threaten ecosystems and livelihoods, but limited management resources mean that managers cannot treat every affected area. This study asks how control efforts can be directed across a landscape to slow an invasion while keeping costs low. We develop a reinforcement-learning framework that can use broader spatial context to control biological invasions, using the spongy moth as a case study. The stochastic simulator captures host-dependent growth, short- and long-distance dispersal, density-dependent mating, and the effects of multiple treatments. The reinforcement-learning strategies are compared with benchmarks that are based on local treatment rules. We find that reinforcement learning achieves the lowest mean cost, with its advantage widening as management complexity increases, such as with simultaneous advancing-front control, nascent-colony management, and established-colony containment on heterogeneous-host landscapes. The reinforcement-learning strategies exhibit highly context-dependent treatment behavior, adjusting treatment choices as invasion dynamics evolve and coordinating interventions across long distances, all of which is learned from scratch.

ecology↗