Unlocking the soundscape of coral reefs with artificial intelligence

Abstract

Passive acoustic monitoring can offer insights into the state of coral reef ecosystems at low-costs and over extended temporal periods. Comparison of whole soundscape properties can rapidly deliver broad insights from acoustic data, in contrast to the more detailed but time-consuming analysis of individual bioacoustic signals. However, a lack of effective automated analysis for whole soundscape data has impeded progress in this field. Here, we show that machine learning (ML) can be used to unlock greater insights from reef soundscapes. We showcase this on a diverse set of tasks using three biogeographically independent datasets, each containing fish community, coral cover or depth zone classes. We show supervised learning can be used to train models that can identify ecological classes and individual sites from whole soundscapes. However, we report unsupervised clustering achieves this whilst providing a more detailed understanding of ecological and site groupings within soundscape data. We also compare three different approaches for extracting feature embeddings from soundscape recordings for input into ML algorithms: acoustic indices commonly used by soundscape ecologists, a pretrained convolutional neural network (P-CNN) trained on 5.2m hrs of YouTube audio and a CNN trained on individual datasets (T-CNN). Although the T-CNN performs marginally better across the datasets, we reveal that the P-CNN is a powerful tool for identifying marine soundscape ecologists due to its strong performance, low computational cost and significantly improved performance over acoustic indices. Our findings have implications for soundscape ecology in any habitat. Author SummaryArtificial intelligence has the potential to revolutionise bioacoustic monitoring of coral reefs. So far, a limited set of work has used machine learning to train detectors for specific sounds such as individual fish species. However, building detectors is a time-consuming process that involves manually annotating large amounts of audio followed by complicated model training, this must then be repeated all over again for any new dataset. Instead, we explore machine learning techniques for whole soundscape analysis, which compares the acoustic properties of raw recordings from the entire habitat. We identify multiple machine learning methods for whole soundscape analysis and rigorously test these using datasets from Indonesia, Australia and French Polynesia. Our key findings show use of a neural network pretrained on 5.2m hours of unrelated YouTube audio offers a powerful tool to produce compressed representations of reef audio data, conserving the datas key properties whilst being executable on a standard personal laptop. These representations can then be used to explore patterns in reef soundscapes using "unsupervised machine learning", which is effective at grouping similar recordings periods together and dissimilar periods apart. We show these groupings hold relationships with ground truth ecological data, including coral coverage, the fish community and depth.