Inadequate sampling of the soundscape leads to overoptimistic estimates of recogniser performance: a case study of two sympatric macaw species

Abstract

Passive acoustic monitoring (PAM) - the use of autonomous recording units to record ambient sound - offers the potential to dramatically increase the scale and robustness of species monitoring in rainforest ecosystems. PAM generates large volumes of data that require automated methods of target species detection. Species-specific recognisers, which often use supervised machine learning, can achieve this goal. However, they require a large training dataset of both target and non-target signals, which is time-consuming and challenging to create. Unfortunately, very little information about creating training datasets for supervised machine learning recognisers is available, especially for tropical ecosystems. Here we show an iterative approach to creating a training dataset that improved recogniser precision from 0.12 to 0.55. By sampling background noise using an initial small recogniser, we can address one of the significant challenges of training dataset creation in acoustically diverse environments. Our work demonstrates that recognisers will likely fail in real-world settings unless the training dataset size is large enough and sufficiently representative of the ambient soundscape. We outline a simple workflow that can provide users with an accessible way to create a species-specific PAM recogniser that addresses these issues for tropical rainforest environments. Our work provides important lessons for PAM practitioners wanting to develop species-specific recognisers for acoustically diverse ecosystems.