Using Artificial Intelligence to Study Protected Species in the Northeast

NOAA Fisheries has formed extensive collaborations to apply machine learning algorithms to identify protected species from images and acoustic recordings

New England/Mid-Atlantic

Whale photos tagged for AI detections — Two whale photos tagged for AI detections. Permit #17355. Credit: NOAA Fisheries

Identifying Right Whales

North Atlantic right whales are one of the most endangered animals on the planet. There are approximately 360 individuals remaining, including fewer than 70 reproductively active females. Human impacts continue to threaten the survival of this species.

Researchers take photographs of these whales from vessels and aircraft, and then compare those photographs to those in the North Atlantic Right Whale Catalog curated by the New England Aquarium. Individual whales can be identified by the pattern on their head along with scars and other markings. Knowing the identity of an individual whale opens up many possible avenues of research and conservation management. These include understanding whale demographics, social structure, reproductive biology, and communication; and launching informed disentanglement operations.

Matching a photograph to the catalog can be time-consuming. NOAA has streamlined this task through innovation, hosting a data science competition on Kaggle and deploying the resulting algorithm on the WildMe Flukebook platform. The resulting system was recently expanded to include the Southern right whale and to allow matching with images taken from vessels. This project was honored with the NOAA Bronze Medal Award in 2019 and the Gears of Government Award in 2020.

Learn more about whale identification

View of the earth's oceans and orbiting satellite. — Satellite in orbit over the Earth’s oceans. Credit: NASA on Unsplash

Detecting Marine Mammals Using Satellites

Monitoring whales is of broad interest to governments, academics, and industry around the world. Scientists use a variety of research platforms—aerial, vessel, passive acoustic—to monitor the abundance and distribution of marine animals and update stock assessments. Each survey platform has its own strengths and challenges. Adding satellites to our current platforms provides us more ways to meet our mission requirements effectively and efficiently.

Several recent publications have demonstrated that identifying whales using very high resolution satellite imagery is technically feasible. NOAA is exploring the development of an operational system in partnership with a broad collaboration including the Naval Research Laboratory, the Bureau of Ocean Energy Management, and the British Antarctic Survey.

Learn more about satellite research.

VIAME tool screenshot for identifying gray seal adults and pups. Shoreline with seals on right side of screen, — Screenshot of the VIAME annotation interface being used to identify gray seal adults and pups. Credit: NOAA Fisheries

Counting Seals

Working with the VIAME (Video and Image Analytics for Marine Environments) software developed by the computer vision experts at Kitware, we have made strides toward automating our process of counting seals from aerial—drone and crewed aircraft—photography. Seal haul-out sites are photographed from overhead passes by manned aircraft or drones, and the resulting images are then annotated in the VIAME software to draw bounding boxes. These annotations are then used to create machine learning models to generate new detections in an active learning process.

This illustration shows the variety of technologies that NOAA Fisheries researchers use to record underwater sounds and study marine animals. The seascape shows bottom- mounted and drifting acoustic recorders, underwater autonomous vehicles, Atlantic cod and humpback whale with tags, and instruments deployed from a NOAA ship and small boat. Colored circles show a zoomed-in view of the instruments and indicate the type of data collected: green for real-time data, orange for archival data, and blue for active — NOAA Fisheries studies marine animals by using a variety of technologies to record underwater sounds, including archival passive acoustic recordings (orange), real-time acoustic data collection (green), and active acoustics (blue). Credit: NOAA Fisheries

Detecting Whale Calls

Acoustic researchers at the Northeast Fisheries Science Center work with other experts to use artificial intelligence to decode the calls of whales. We have collected years of recordings containing whale calls using various technologies. Computers are faster than humans when it comes to sorting through this volume of data to pull out the meaningful sounds, and identifying what animal is making that sound and why. We are working on programs to allow us to use machine learning to help identify, localize, and visualize marine mammal communication.

Leatherback turtle view at sealevel looking over his right shoulder. — Video image of surfacing leatherback turtle. Credit: NOAA Fisheries, ESA Permit 22218

Documenting Turtle Behavior

We estimate sea turtle population numbers from aerial observations. Not all turtles are on the surface when the observations are made, so it is important to estimate the percentage of time turtles spend on the surface. Over the past few years, turtle ecologists at the Northeast Fisheries Science Center and partner organizations have temporarily attached video cameras to the backs of leatherback turtles and recorded hours of video footage. We are using artificial intelligence to analyze that video footage and extract behavior information that can be used to improve turtle population estimates.

More Information

Last updated by Northeast Fisheries Science Center on 02/02/2024