Deep learning workflow to support in-flight processing of digital aerial imagery for wildlife population surveys

By: , and 



Deep learning shows promise for automating detection and classification of wildlife from digital aerial imagery to support cost-efficient remote sensing solutions for wildlife population monitoring. To support in-flight orthorectification and machine learning processing to detect and classify wildlife from imagery in near real-time, we evaluated deep learning methods that address hardware limitations and the need for processing efficiencies to support the envisioned in-flight workflow. We developed an annotated dataset for a suite of marine birds from high-resolution digital aerial imagery collected over open water environments to train the models. The proposed 3-stage workflow for automated, in-flight data processing includes: 1) image filtering based on the probability of any bird occurrence, 2) bird instance detection, and 3) bird instance classification. For image filtering, we compared the performance of a binary classifier with Mask Region-based Convolutional Neural Network (Mask R-CNN) as a means of sub-setting large volumes of imagery based on the probability of at least one bird occurrence in an image. On both the validation and test datasets, the binary classifier achieved higher performance than Mask R-CNN for predicting bird occurrence at the image-level. We recommend the binary classifier over Mask R-CNN for workflow first-stage filtering. For bird instance detection, we leveraged Mask R-CNN as our detection framework and proposed an iterative refinement method to bootstrap our predicted detections from loose ground-truth annotations. We also discuss future work to address the taxonomic classification phase of the envisioned workflow.

Study Area

Publication type Article
Publication Subtype Journal Article
Title Deep learning workflow to support in-flight processing of digital aerial imagery for wildlife population surveys
Series title PLoS ONE
DOI 10.1371/journal.pone.0288121
Volume 19
Issue 4
Year Published 2024
Language English
Publisher PLOS
Contributing office(s) Upper Midwest Environmental Sciences Center
Description e0288121, 19 p.
Country United States
State Massachusetts, Wisconsin
County Manitowoc County
Other Geospatial Lake Michigan, Nantucket Shoals area
Google Analytic Metrics Metrics page
Additional publication details