Unsupported Browser Detected

Internet Explorer lacks support for the features of this website. For the best experience, please use a modern browser such as Chrome, Firefox, or Edge.

Optics Technology: At the Heart of Marine Research

January 14, 2025

Videos and images from advanced underwater cameras are a critical component of collecting data. We use this information to make informed management decisions for marine species and habitat.

Flower Garden Banks Reef Scene A Spanish hogfish (left) and yellowtail damselfish (right) peek out from a reef in the Gulf of Mexico. Credit: Emma Hickerson/NOAA

At NOAA Fisheries, optics technology plays a major role in gathering data to better understand our marine and coastal ecosystems. Optics uses tools such as advanced underwater cameras to generate video and images that scientists use to better understand biodiversity and track the behavior of species. 

To collect data, many surveys have traditionally relied on securing time on a NOAA ship. Researchers typically sail to the area where the survey is taking place, but they are limited to collecting data in areas the ship can navigate. Ship time can be very costly, and researchers must plan their efforts around the time of year the ship is available. 

Optics increases our capacity to help gather this same data by using video and image data instead of human observation. This is particularly helpful in the deepest parts of the ocean, which can be difficult to access due to a lack of light and extreme pressure. We are able to “see” the underwater ecosystems we protect while ensuring safer operations. Incorporating this information into our research provides a richer understanding of precious marine resources.

The findings have a direct impact on management decisions for habitat and species conservation. In warmer climates, we use optics to survey corals in the National Coral Reef Monitoring Program. In science centers across the country, we use remote underwater video cameras to identify marine resources.

In colder climates such as remote locations of Alaska, we identify marine mammals with drone footage and infrared optics. And in Antarctica, we attach cameras to autonomous underwater vehicles such as gliders to gather information on krill and plankton.

Data Challenges

Image
A clip taken from one of the stationary video surveys in the Gulf of Mexico.
Video footage from the Gulf Fishery Independent Survey of Habitat and Ecosystem Resources survey. Reviewing and labeling data from the footage can be a lengthy process. Credit: NOAA Fisheries

As researchers increasingly rely on optics, however, they are faced with a growing problem: The videos and images they collect are generating massive amounts of data. Dr. Matthew Campbell, Gulf and Caribbean Reef Fish Branch Chief for the Southeast Fisheries Science Center, shared his insights into the challenge:

“Once a survey is complete, we will have collected thousands of hours of footage that must be reviewed. Many surveys deploy hundreds—or even thousands—of cameras. The sheer amount of data can be overwhelming.”

For many surveys, transforming raw data into usable information can take a very long time. Each data point must be manually annotated for identification and all fish counts must be recorded. The process often consists of correctly tagging thousands of data points in a video or image one at a time.

A spherical device equipped with cameras on all sides is used for some surveys. It is dropped to the ocean floor to record fish and their surroundings in every direction. Credit: NOAA Fisheries
A spherical device equipped with cameras on all sides is used for some surveys. It is dropped to the ocean floor to record fish and their surroundings in every direction. Credit: NOAA Fisheries

Each survey team has a limited amount of resources available, especially in terms of personnel, so a bottleneck can occur during the post-processing phase. Once the annotation stage is complete, researchers can then start the process of analyzing the data, which can also be time-consuming. Fully processing and analyzing optical data can take a year or longer, depending on the amount of data collected.

Once collected, data storage can also be a challenge. Many researchers opt to store the data locally, which means they are limited to the local network’s computational power. In addition, only those who have access to the local network can work on the dataset. As a result, the data are not available for others to help with the analysis process or to conduct related scientific research. 

Of course, working with our survey data requires highly technical skills. There is limited flexibility on which software to use. Many researchers specializing in fisheries biology must first learn technical programming languages and software before they analyze any data. While user abilities range from entry-level to expert-level, more personnel with advanced computational skillsets are needed.

To address current challenges around data collection and with optical processing and analysis, we are focusing our investments in three main categories: data acquisition, powerful processing tools, and increased accessibility. 

Data Acquisition

Image
Sail drone in water with mountains in background.
Sail drone gliding over frigid waters near the Alaskan mountains. Pairing optics with uncrewed systems allows for increased coverage of areas that are challenging to access. Credit: NOAA Fisheries

We will use small, low-cost optical devices that can run on multiple platforms to collect imagery. By integrating these devices into our uncrewed systems already in place, we can cover areas that ships cannot. Dr. Campbell explained: 

“Securing ship time is a challenging process. We carefully plan our work out at sea within the limited ship time given for a specific survey. With optics technology, we are reducing our reliance on NOAA ships by using uncrewed systems such as remotely deployed gliders.”

In regions that have had limited data collection capabilities, we will start collecting data. High-resolution satellite imagery will expand our survey range to these regions. The aerial imagery will be validated using uncrewed, ship-based, and aircraft-based systems.

Powerful Processing Tools

Image
AI and machine learning models provide researchers with advanced data analysis methods that reduce manual processing time. Credit: NOAA Fisheries
AI and machine learning models provide researchers with advanced data analysis methods that reduce manual processing time. Credit: NOAA Fisheries

We will use a cloud-based system to handle our large datasets efficiently. This will allow us to collaborate remotely and ramp up computational needs without investing in costly physical infrastructure. We will also improve the tools we already use to make sure they handle data formats from all fisheries science centers. 

We will process data more efficiently through AI, machine learning, and edge-computing capabilities that automatically analyze images. This saves time, reduces resources, and leaves more availability for other projects. 

Increased Accessibility

We will develop world-leading classifiers, detectors, trackers, and segmentation models for marine life. Our tools and labeled optical image data sources will be accessible to our scientists across diverse regions. Easier access via web-based portals will help data transform quickly into products that inform management decisions across the country.

On the personnel side, we are hiring data scientists and engineers to create easy-to-use pathways. For our current staff, we will create and share resources that make it more efficient to learn, experiment, and adapt models for specific applications. Along with participating in user groups and rotational assignments, this will allow staff to increase their analyst and developer skills.

Optics in Action

Image
A camera's view underwater shows a grouper and several other fish, outlined and identified to species or genus by an automated computer program
Example of automated image analysis software. It shows colored boxes around detected fish with the software assigned name and estimated confidence level of the identification. Species in the image include red snapper, Warsaw grouper, and spotfin hogfish. Credit: NOAA Fisheries

Some of these efforts to modernize our optics technology are already underway. One such case is the Gulf Fishery Independent Survey of Habitat and Ecosystem Resources survey, which started field work in 2020. Currently funded for 10 years, it brings together three historic surveys in the Gulf region under a single, comprehensive habitat-based design. Dr. Campbell highlighted the success of this project:

“We have an automated processing system that is cloud-based, with automated workflows and shared technical knowledge. This kind of system benefits not only the scientists directly working on a project, but also those beyond the laboratories where the data were created.”

Every year, this large-scale project deploys around 2,000 cameras and maps about 3,000 linear nautical miles. The survey results are critical to help assess the state of managed reef fish throughout the Gulf of Mexico. This time series of camera data over 30 years is used to estimate fish abundance and habitat composition in the northern Gulf of Mexico. 

A key component of the survey involves collecting three types of stationary video data: single camera view, 360-degree view, and stereo view. The cameras are baited, which encourages fish to explore nearby and bring them close to the cameras. The single camera and 360-degree camera views allow us to identify fish species and abundance, while the stereo camera view helps estimate fish length. Using this optical data, we then leverage machine learning to support the fish counting process.

We train computer models for fish detection and classification using open-source software and data from our library of manually annotated images and videos. Optical conditions, fish density, and habitat complexity can have an impact on the model’s accuracy. This method allows us to quickly generate semi-automated fish counts, which helps to reduce human effort and results in a more efficient fish counting process.

The Future of Optics

Looking ahead, we will continue harnessing the power of optics technology. Through Inflation Reduction Act funding, we will be conducting coupled video-acoustic surveys to improve data collection of reef fish—specifically red snapper—in the Gulf of Mexico. We will use a combination of underwater video, imaging sonar, split-beam echosounder, and x-ray data. We will develop new methods of data collection and automation to help us better understand the full scope of red snapper populations in the Gulf. These projects are scheduled for the first half of 2025.

The future of marine research is closely connected to the continued advancement of optics technology. By investing in innovative tools and collaborative efforts, we can address the challenges of data collection and analysis while unlocking new possibilities for understanding and protecting our oceans. With these efforts, we are not only improving the way we study marine life, but also helping to preserve our vital marine resources.

Last updated by Office of Communications on February 03, 2025