Linking automated image analysis to ecological inference: high-throughput monitoring of soil fauna
1 - Automated in situ sensors - e.g., buried scanners - are transforming biodiversity monitoring by generating data at spatio-temporal resolutions unattainable through traditional sampling, including in cryptic environments such as soil that have remained largely inaccessible to existing methods. However, extracting ecologically meaningful information from these data streams requires substantial image processing effort that currently constitutes a critical bottleneck, particularly when the signal-to-noise ratio is low and annotated training data are scarce. 2 - Standard end-to-end deep learning detection pipelines offer unsatisfactory results due to the lack of training data and heterogeneity of the taxa of interest. We explore the potential of combining traditional computer vision algorithms with state-of-the-art deep learning models to build an efficient raw data processing pipelines from limited annotation effort. Specifically, based on the observation that the background barely changes, we focus on the differences between two consecutive images to turn the initial detection problem (with very low signal) into a simpler classification problem, which we solve by fine-tuning foundation models on limited annotated data. 3 - Our approach significantly reduces the annotation effort, allowing us to release an open dataset with about 600 soil scans and more than 8 000 labeled invertebrate occurrences across nine taxa. Using this dataset to train our models, we obtained population count estimates with relative errors ranging from 10% to 61% across taxa over a three-month period. Ecological validation through a land-use stability analysis showed full directional congruence between automated and expert-annotated classifications across all nine taxa examined, with effect-size discrepancies proportional to per-taxon classification accuracy. 4 - These results demonstrate that combining domain-specific heuristics with fine-tuned foundation models provides an effective and data-efficient strategy for automating ecological image processing workflows in low-signal, data-scarce contexts. The validated pipeline removes the manual annotation bottleneck that has historically limited scanner-based soil monitoring to short observational windows and restricted taxonomic scope, opening the way for continuous, large-scale tracking of soil invertebrate community dynamics at resolutions previously unachievable.