Automated bird sound classifications of long-duration recordings produce occupancy model outputs similar to manually annotated data

ABSTRACT Occupancy modeling is used to evaluate avian distributions and habitat associations, yet it typically requires extensive survey effort because a minimum of 3 repeat samples are required for accurate parameter estimation. Autonomous recording units (ARUs) can reduce the need for surveyors on-site, yet their utility was limited by hardware costs and the time required to manually annotate recordings. Software that identifies bird vocalizations may reduce the expert time needed if classification is sufficiently accurate. We assessed the performance of BirdNET—an automated classifier capable of identifying vocalizations from >900 North American and European bird species—by comparing automated to manual annotations of recordings of 13 breeding bird species collected in northwestern California. We compared the parameter estimates of occupancy models evaluating habitat associations supplied with manually annotated data (9-min recording segments) to output from models supplied with BirdNET detections. We used 3 sets of BirdNET output to evaluate the duration of automatic annotation needed to approach manually annotated model parameter estimates: 9-min, 87-min, and 87-min of high-confidence detections. We incorporated 100 3-s manually validated BirdNET detections per species to estimate true and false positive rates within an occupancy model. BirdNET correctly identified 90% and 65% of the bird species a human detected when data were restricted to detections exceeding a low or high confidence score threshold, respectively. Occupancy estimates, including habitat associations, were similar regardless of method. Precision (proportion of true positives to all detections) was >0.70 for 9 of 13 species, and a low of 0.29. However, processing of longer recordings was needed to rival manually annotated data. We conclude that BirdNET is suitable for annotating multispecies recordings for occupancy modeling when extended recording durations are used. Together, ARUs and BirdNET may benefit monitoring and, ultimately, conservation of bird populations by greatly increasing monitoring opportunities. LAY SUMMARY Occupancy modeling provides valuable information for understanding bird distributions, but often requires extensive survey effort. Autonomous recording units (ARUs) produce vast amounts of data, yet manually identifying birds on recordings is time-consuming. We evaluated the performance of an automated bird sound classifier, BirdNET, by comparing occupancy models that used manually and BirdNET-annotated data for 13 species in northwestern California, USA. We manually identified bird species heard during 9-min recordings at 34 sites, and used BirdNET to identify birds during 9–260-min recordings from each site. We also manually verified 100 BirdNET detections for each species. BirdNET correctly identified most bird species detected during manual bird identification when data were restricted respectively to nearly all (90% correct) or only high confidence (65% correct) detections. Habitat associations were similar across all models. We conclude that BirdNET is a useful tool for automatic annotation of bird vocalization data needed to model bird presence or absence.