Hopping, W. A. | Search bioRxiv

bioRxiv2025

The reliability of acoustic classification models for determining avian vocalisation patterns

Automated detection and classification of species vocalisations offers the potential to utilise acoustic datasets across unprecedented spatial and temporal scales. However, classification algorithms inevitably generate errors, and error rates vary with context. While methods for quantifying error rates in ecoacoustics are well established, there is limited research on what level of model performance is sufficient to reliably determine avian vocalisation patterns. Using an extensive fully expert-labelled acoustic dataset from Peru (18 hours, 6 sites), we examined changes in the probability of detecting target bird vocalisations in the first hour after dawn to address three key questions: (1) How sensitive are models predicting detection probability over time to reductions in classification accuracy? (2) To what extent does aggregating detections over longer time periods impact classification accuracy? Additionally, we used the labelled dataset to assess how the creation and composition of a test dataset for assessing classifier performance can impact the reliability of accuracy metrics: (3) Are estimates of classification precision robust when test and deployment datasets are not independent and identically distributed? Our results indicate that poor classification performance--especially low precision--can lead to misleading inferences about temporal patterns of vocalisations. Aggregating classifier predictions over longer time periods improved recall but often resulted in misleading patterns of vocal behaviour by reducing precision and temporal resolution. We also demonstrate that precision can be substantially overestimated when species presences are rarer in the deployment dataset than in test data. These findings highlight the importance of cautious application of automated classification in acoustic ecology and the need for accuracy assessment methods tailored to the intended ecological analysis.

ecology↗

bioRxiv2021

Simultaneous acoustic monitoring uncovers evidence of biodiversity loss and overlooked temporal variation in a threatened Amazonian bird community

The vocal activity and detectability of tropical birds are subject to high levels of temporal heterogeneity, but quantifying patterns of diel and day-to-day variation in complex systems is challenging with traditional point count methods. As a result, research concerning stochastic temporal effects on tropical avian assemblages is limited, typically offering only broad conclusions, e.g., overall activity is highest in the first few hours of the morning and some species are active at different times of the day. Passive acoustic monitoring introduces several advantages for studying temporal variation, particularly by enabling simultaneous and continuous data collection across adjacent sites. Here, we employed autonomous recording units to quantify temporal variation in avian vocal activity and observed species richness at an Amazonian reserve in Madre de Dios, Peru--a region featuring some of Earths richest, most complex avian assemblages. We manually annotated 18 dawn hour recordings, collected simultaneously from three separate days at the same six sites, which represent various microhabitats and avian community compositions. We documented significant and consistent temporal variation in avian vocal activity levels and observed species richness within the dawn hour and across days. We found that temporal effects were stronger for vocal activity than for observed species richness and that vocal activity patterns over the course of the dawn hour varied on a species-to-species basis. Our results indicate that overlooked temporal variation in Amazonian soundscapes may obfuscate the results of surveys that fail to sufficiently account for temporal variables with simultaneous monitoring. While manual analysis of large volumes of soundscape data remains challenging, such data should be collected as a supplement to traditional surveys whenever possible. Rapidly advancing research concerning the automated processing of acoustic data could lead to more efficient methods for reducing temporal bias and improving the calibration and accuracy of ornithological surveys in the tropics.

ecology↗