Brostow, G. | Search bioRxiv

bioRxiv2018

VideoTagger: User-Friendly Software for Annotating Video Experiments of Any Duration

BackgroundScientific insight is often sought by recording and analyzing large quantities of video. While easy access to cameras has increased the quantity of collected videos, the rate at which they can be analyzed remains a major limitation. Often, bench scientists struggle with the most basic problem that there is currently no user-friendly, flexible, and open source software tool with which to watch and annotate these videos.\n\nResultsWe have created the VideoTagger tool to address these and many of the other associated challenges of video analysis. VideoTagger allows non-programming users to efficiently explore, annotate, and visualize large quantities of video data, within their existing experimental protocols. Further, it is built to accept programmed plugins written in Python, to enable seamless integration with other sophisticated computer-aided analyses.\n\nWe tested VideoTagger ourselves, and have a growing base of users in other scientific disciplines. Capitalising on the unique features of VideoTagger to play back infinite lengths of video footage at various speeds, we annotated 39h of a Drosophila melanogaster lifespan video, at approximately 10-15x faster than real-time. We then used these labels to train a machine-learning plugin, which we used to annotate an additional 538h of footage automatically. In this way, we found that flies fall over spontaneously with increasing frequency as they age, and also spend longer durations struggling to right themselves. Ageing in flies is typically defined by length of life. We propose that this new mobility measure of ageing could help the discovery of mechanisms in biogerontology, refining our definition of what healthy ageing means in this extremely small, but widely used, invertebrate.\n\nConclusionsWe show how VideoTagger is sufficiently flexible for studying lengthy and/or numerous video experiments, thus directly improving scientists productivity across varied domains.

animal behavior and cognition↗

bioRxiv2017

Bat Detective - Deep Learning Tools for Bat Acoustic Signal Detection

O_LIPassive acoustic sensing has emerged as a powerful tool for quantifying anthropogenic impacts on biodiversity, especially for echolocating bat species. To better assess bat population trends there is a critical need for accurate, reliable, and open source tools that allow the detection and classification of bat calls in large collections of audio recordings. The majority of existing tools are commercial or have focused on the species classification task, neglecting the important problem of first localizing echolocation calls in audio which is particularly problematic in noisy recordings.\nC_LIO_LIWe developed a convolutional neural network (CNN) based open-source pipeline for detecting ultrasonic, full-spectrum, search-phase calls produced by echolocating bats (BatDetect). Our deep learning algorithms (CNN FULL and CNN FAST) were trained on full-spectrum ultrasonic audio collected along road-transects across Romania and Bulgaria by citizen scientists as part of the iBats programme and labelled by users of www.batdetective.org. We compared the performance of our system to other algorithms and commercial systems on expert verified test datasets recorded from different sensors and countries. As an example application, we ran our detection pipeline on iBats monitoring data collected over five years from Jersey (UK), and compared results to a widely-used commercial system.\nC_LIO_LIHere, we show that both CNNFULL and CNNFAST deep learning algorithms have a higher detection performance (average precision, and recall) of search-phase echolocation calls with our test sets, when compared to other existing algorithms and commercial systems tested. Precision scores for commercial systems were reasonably good across all test datasets (>0.7), but this was at the expense of recall rates. In particular, our deep learning approaches were better at detecting calls in road-transect data, which contained more noisy recordings. Our comparison of CNNFULL and CNNFAST algorithms was favourable, although CNNFAST had a slightly poorer performance, displaying a trade-off between speed and accuracy. Our example monitoring application demonstrated that our open-source, fully automatic, BatDetect CNNFAST pipeline does as well or better compared to a commercial system with manual verification previously used to analyse monitoring data.\nC_LIO_LIWe show that it is possible to both accurately and automatically detect bat search-phase echolocation calls, particularly from noisy audio recordings. Our detection pipeline enables the automatic detection and monitoring of bat populations, and further facilitates their use as indicator species on a large scale, particularly when combined with automatic species identification. We release our system and datasets to encourage future progress and transparency.\nC_LI

ecology↗