Pre-implementation of EM/ER in the North Pacific

1. Project Description

1.1. Background

Electronic monitoring (EM) technologies offer a way to obtain independent fishery data onboard vessels where space is limited and/or safety is a concern for human observers. The Fisheries Monitoring and Analysis Division (FMA), located at the Alaska Fisheries Science Center, is primarily responsible for collecting timely and high quality fisheries dependent data, which are used to manage the groundfish stocks in the Bering Sea, Aleutian Islands and Gulf of Alaska. Historically the FMA Division has used observers on fishing vessels to collect biological and catch composition data used to estimated discard. This information is critical to inform fisheries management decisions and development of annual stock assessments.

In January 2013, the Alaska observer program was restructured to include observer coverage on vessels less than 60 feet in length in the North Pacific groundfish and Pacific halibut fisheries. A small percentage of these vessels will be unable to accommodate an observer primarily due to vessel size or space available for observer sampling duties. At the request of the Council, NMFS presented an EM/ER strategic plan (Loefflad et al., 2014, Appendix A) to the Council In June, 2013. The document provided a vision for integrating electronic technologies into the North Pacific fisheries-dependent data collection program.

The Council adopted the strategic plan as a guidance document for incorporating EM into the Observer Program. In addition, the Council recognized the halibut and sablefish fisheries as the highest priority for integration of EM and they recommended use of a catch estimation approach to develop EM for these fisheries. The Council also created an EM Workgroup and tasked it to: identify EM performance standards, operational procedures, and sampling and deployment plans appropriate for IFQ vessels and also look at implementation vehicles and potential phase-in approaches. The Council recommended that the EM Workgroup use the following sections of the strategic plan to focus its efforts to develop a catch estimation based program for the IFQ fisheries: Goal II, Objective 1, Strategy C and Goal III, Objective 1, Strategy A (Loefflad, 2014).

Most recently, Council motion at the October 2014 meeting endorsed a target of taking the first steps at operationalizing EM in the small vessel strata in 2016. The Council also supported the work of the EMWG to continue development of projects identified in the (Cooperative Research Plan) CRP tables, using the subgroup approach, with the goal of providing a document for SSC review at the February 2015 meeting.

In April 2013 the National Marine Fisheries Service (NMFS) implemented a new policy encouraging the development of electronic technologies for fishery dependent data collection to complement or improve existing data collection programs. The goal is to achieve the most cost-effective and sustainable approach that ensures alignment of management goals, data needs, funding sources, and regulations. This project addresses the Electronic Monitoring Technology Development area of interest specified by the FIS Request for Proposals (RFP) and the FIS Project Management Team (PMT) for FY 2014 “Develop and promote tools to facilitate timely reporting of fisheries information” by automating and expediting video imagery review and analysis.

The project also addresses NMFS’ policy directive on Electronic Technologies and Fishery-Dependent Data Collection; Objective 5. “NOAA Fisheries encourages the use of electronic technologies that utilize open source code or standards that facilitate data integration and offer long-term cost savings rather than becoming dependent on proprietary software.”

1.2. History

The Observer program has completed or nearly completed two relatively small scale FIS projects “Improvement of process for image analysis and image storage” and “Electronic logbook data entry and transmissions to improve timeliness and accuracy of dependent fishery data collection”. These two projects supported initial development of application to integrate EM data into NORPAC, integrate sensor data with self-reported information (e-log) and purchase laptops for deployment. There are also two ongoing funded projects to advance EM/ER implementation into North Pacific fisheries: one funded by FIS “Automated Image processing for Fisheries Applications” and one funded by NOP “Integrating GPS and Sensor data with e-logbooks to collect temporal spatial catch and effort data in the small boat commercial groundfish and halibut fleet”. Funding for both of these projects are administered through PSMFC that has recently became available to hire contract personnel for application development, purchase sensor hardware to support e-logbook application development and hire field personnel to support testing and development.

The field position is already filled and a contract position for application development will be coming on board within the next month. A second contract position for application development for image classification with specific skills using openCV will be hired January, 2014. In 2015, EM/ER will be deployed and tested on a variety of vessels through the PSMFC service provider contract, at-sea samplers and field staff. The platforms will include survey vessels (IPHC, Sablefish, potentially trawl), a contracted fixed gear vessel (Pot and setline) and volunteer vessels that are part of the CRP envisioned by the Council’s EM Work Group. An EM system will be installed on the contract vessel in the next month, on up to 15 volunteer vessels beginning March and survey vessels thereafter. The overall objective is to evaluate the efficacy of EM/ER to support catch estimation and testing of EM/ER equipment to support pre-implementation of EM/ER on a test fleet in 2016 per Councils October 2014 motion.

To address this challenge, the FMA Division is seeking funding through this proposal for continue funding of contract personnel for deploying and testing EM/ER equipment and further development of applications on track into 2016. This funding would help facilitate the adoption of EM on smaller vessels subject to observer coverage but unable to accommodate observers due to space limitations. The primary role of software development is two-fold including: software and procedures to improve and automate the video analysis and storage of specific image events to provide a cost effective collection of fisheries dependent information and to integrate EM data into catch accounting.

1.3. Overview

There have been over 60 EM studies completed over the last 15 years and all of the video collected require manual review of video and still image data to extract meaningful information. Automated image processing has the potential to greatly reduce the time necessary for manual analysis, further improving the value of image-based sampling approaches. While automated image processing methods are well established in biomedical and security applications, software packages capable of automated target detection and identification of fish are not commercially available.

This project proposes to develop automatic detection, sizing, and classification of fish targets from stereo-video imagery of fish passing on a conveyor belt or sliding on a chute. The project involves controlling image acquisition, developing and applying computer algorithms for image processing, and providing user interfaces and suitable data outputs for operation of software by fisheries biologists. Tasks can be accomplished by applying and modifying classification algorithms developed in computer vision industry, with improvements and adjustments for the specific challenges of fish imagery.

This project also proposes to integrate EM data collection into the Observer database (NORPAC) that could eventually be used in catch estimation. This will be the first time that data resulting from EM collection is used as scientific data for estimation and is a huge step in the evolution of EM/ER.

This work will provide data to enhance and complement the ongoing research for EM/ER development across the country and in numerous fisheries. In addition, the data and products/deliverables will be developed in open source software code that can be shared with all other NMFS FMC’s, FIN’s, and any other NMFS partners. Also the deliverables will be used by the staff of the North Pacific Fisheries Management Council in assisting with the development of EM monitoring programs.

1.4. Objectives

This project directly relates to the FIS vision to develop and promote tools to facilitate timely reporting of fisheries information and the NOP objective to Develop and support national standards and policies to create high quality, cost effective, efficient, and productive observer programs. An automated species recognition application addresses cost issues identified with currently available EM systems, mainly the prohibitive costs associated manual review of image data. This project is integral for support the timeline of EM/ER implementation in the North Pacific to further work from previous projects.

The overall goal of this project is to provide field-tested methods to provide quantifiable image-based data from fisheries with stereo camera and camera chute based sampling systems. There are two primary objectives of the work including: 1) build on current software applications for integrating EM/ER data for catch estimation and 2) develop new software to automatically detect individual fish in video data streams from machine vision cameras placed above conveyor belts or fish chutes, capture stereo image pairs of fish targets, determine fish size, and identify fish targets to species. The resulting software products should be built as executable programs with user settable parameters.

1. Integration of EM data

This work will provide for integration of EM/ER data into the current NORPAC dataset to operationalize catch estimation similar to observer data now being collected. The application will involve much testing and development to ensure seamless integration into catch accounting.

2. Fish Detection

This work will improve upon and operationalize existing camera control software for fish detection as fish pass at different times under a set of stereo-cameras or a camera chute. One camera will be used to evaluate fish presence, and when a fish is present a stereo image pair will be captures and save to disk. The application will involve testing to determine optimal settings and evaluate possibility of taking rapid stereo-camera sequences and selecting best image pair for further analysis. The hardware systems, including cameras, strobes, and prototype fish chute is under development at AFSC. Algorithms developed should be able to separate two or more fish that are potentially touching, but not overlapping.

3. Size estimation

Stereo-camera sets or chute camera will be calibrated using OpenCV or Matlab approaches, and the length of fish targets estimated as precisely as possible, including potential corrections for curved body position. Stereo-camera arrangement will remove necessity of exact placement of cameras relative to fish passage surface, making the system more flexible for future installations.

4. Automated species classification

Fish targets that are detected and separated for the image background will be passed on to a classification algorithm that will determine class membership of up to 20 different classes of fish. The classifications should be made in a probabilistic framework, allowing users to incorporate classification uncertainty in to data analysis. A potential hierarchical structure can be applied to group fish with low species level classification confidence in species groups of similar appearing fish types. Targets with high uncertainty can be identified for manual review.

This project was in response to the Electronic Monitoring Development and Implementation (FIS and National Observer Program) RFP.

7600 Sand Point Way NE, Building 4