NOAA Fisheries has formed extensive collaborations to apply machine learning algorithms to identify protected species from images and acoustic recordings
North Atlantic right whales are one of the most endangered animals on the planet; the latest preliminary estimate suggests there are fewer than 350 left. Researchers take photographs of these whales from vessels and aircraft, and then compare those photographs to those in the North Atlantic Right Whale Catalog curated by the New England Aquarium. Individual whales can be identified by the pattern on their head along with scars and other markings. Knowing the identity of an individual whale opens up many possible avenues of research and conservation management. These include understanding whale demographics, social structure, reproductive biology, and communication; and launching informed disentanglement operations.
Matching a photograph to the catalog can be time-consuming. NOAA has streamlined this task through innovation, hosting a data science competition on Kaggle and deploying the resulting algorithm on the WildMe Flukebook platform. The resulting system was recently expanded to include the Southern right whale and to allow matching with images taken from vessels. This project was honored with the NOAA Bronze Medal Award in 2019 and the Gears of Government Award in 2020.
Learn more about whale identification
Monitoring whales is of broad interest to governments, academics, and industry around the world. Scientists use a variety of research platforms—aerial, vessel, passive acoustic—to monitor the abundance and distribution of marine animals and update stock assessments. Each survey platform has its own strengths and challenges. Adding satellites to our current platforms provides us more ways to meet our mission requirements effectively and efficiently.
Several recent publications have demonstrated that identifying whales using very high resolution satellite imagery is technically feasible. NOAA is exploring the development of an operational system in partnership with a broad collaboration including the Naval Research Laboratory, the Bureau of Ocean Energy Management, and the British Antarctic Survey.
Learn more about satellite research.
Working with the VIAME (Video and Image Analytics for Marine Environments) software developed by the computer vision experts at Kitware, we have made strides toward automating our process of counting seals from aerial—drone and crewed aircraft—photography. Seal haul-out sites are photographed from overhead passes by manned aircraft or drones, and the resulting images are then annotated in the VIAME software to draw bounding boxes. These annotations are then used to create machine learning models to generate new detections in an active learning process.
Acoustic researchers at the Northeast Fisheries Science Center work with other experts to use artificial intelligence to decode the calls of whales. We have collected years of recordings containing whale calls using various technologies. Computers are faster than humans when it comes to sorting through this volume of data to pull out the meaningful sounds, and identifying what animal is making that sound and why. We are working on programs to allow us to use machine learning to help identify, localize, and visualize marine mammal communication.
We estimate sea turtle population numbers from aerial observations. Not all turtles are on the surface when the observations are made, so it is important to estimate the percentage of time turtles spend on the surface. Over the past few years, turtle ecologists at the Northeast Fisheries Science Center and partner organizations have temporarily attached video cameras to the backs of leatherback turtles and recorded hours of video footage. We are using artificial intelligence to analyze that video footage and extract behavior information that can be used to improve turtle population estimates.
North Atlantic right whale Catalog #3560 ‘Snow Cone’ sighted December 2, 2021 entangled and with a new calf. Photo credit: Florida Fish and Wildlife Conservation Commission taken under NOAA permit 20556.
Four North Atllantic right whales.
North Atlantic right whale mother and calf. Credit: Clearwater Marine Aquarium Research Institute, taken under NOAA permit #20556-01.