Transitioning to New Recreational Fishing Survey Designs
NOAA Fisheries has established procedures for reviewing, certifying, and transitioning to new or improved recreational data collection methods.
NOAA Fisheries’ Marine Recreational Information Program follows an ongoing cycle of continuous improvement: Existing methods are evaluated, potential improvements are tested and reviewed, and new or improved recreational fishing survey designs are put in place. For data derived from new or improved methods to support fisheries science and management, these methods must be sound, and the catch and effort estimates they produce must be incorporated into the existing time series with minimal disruption.
NOAA Fisheries fosters continued methodological improvement while maintaining the integrity of our long-term recreational catch and effort estimates through its established policy and procedures for reviewing, certifying, and transitioning to new or improved survey designs. These policies and procedures are summarized on this webpage and described in more detail in the following documents:
- Policy Directive 04-114: Implementing Recreational Fishery Catch and Effort Survey Design Changes (PDF, 4 pages)
- Procedural Directive 04-114-01: Guidance and Procedures for the Transition Process for Modification of Recreational Fishing Catch and Effort Methods (PDF, 5 pages)
- Procedural Directive 04-114-02: Guidance and Procedures for the MRIP Certification Process (PDF, 10 pages)
Survey sponsors may contact the Marine Recreational Information Program's Program Management Team for more information.
NOAA Fisheries’ process for reviewing the statistical rigor of a recreational fishing survey design is known as certification. To earn certification, survey designs and estimation methods must be documented, adhere to applicable standards, and earn validation by a peer review. The steps of the certification process are described in more detail in Procedural Directive 04-114-02 (PDF, 10 pages).
The certification of a new or improved survey design means:
- A peer review has found the survey design to be a statistically valid approach to meet survey objectives and provide key estimates.
- The program is eligible to receive financial support from NOAA Fisheries. For surveys that pre-date MRIP and our certification process and are already receiving NOAA Fisheries funding, becoming certified allows them to remain a priority to receive continued funding.
- Certification does not guarantee implementation or use in federal fisheries management. Decisions as to how the data are used are made by NOAA Fisheries and partners, independent of the certification process.
Survey designs and estimation methods must be documented and pilot tested before the certification process begins. Following certification, survey sponsors must submit annual reports to confirm the design is being implemented as originally described. These documentation requirements are the same documentation requirements described in Recreational Fishing Survey and Data Standards 1, 2, 3, and 5.2 (Survey Concepts and Justification, Survey Design, Data Quality, and Annual Reporting).
Documentation submitted by survey sponsors can be found in the MRIP Reports Database.
Certified survey designs and estimation methods must meet applicable standards and earn validation by a peer review. Independent reviewers, who are experts in the field of survey statistics, must consider the following criteria in conducting their comprehensive evaluation:
- Sample survey designs follow formal probability sampling protocols, with known inclusion probabilities at all sampling stages.
- Estimation methods appropriately weight sample data to account for the sampling design. Both point and variance estimates are produced.
- Methods are in place to measure and/or correct for potential biases due to under-coverage, non-response, and/or response errors.
- The sensitivity of the accuracy of the survey to potential sampling and non-sampling errors is understood, and measures to evaluate, reduce, and/or limit that sensitivity are described.
- The sensitivity of the survey design to potential implementation errors is documented, and measures to evaluate, reduce, and/or limit that sensitivity are described.
- New survey design and/or estimation methods are compared to the design and/or methods they will replace, as well as any other certified survey components currently used to estimate the same population parameters. The relative statistical validity and efficiency of each are described.
- The survey design and/or estimation methods are collecting data and/or producing information that meet science and management needs.
Certified survey designs and estimation methods must receive a recommendation for approval from the Marine Recreational Information Program’s Executive Steering Committee and final approval from NOAA Fisheries’ Director of Scientific Programs. These decisions are informed by the outcome of the peer review.
Because stock assessments require consistent, long-term time series of recreational catch, new recreational fishing surveys cannot be incorporated into the cycle of science and management until differences between old and new estimates are examined and accounted for. For this reason, NOAA Fisheries requires the sponsors of any new or improved survey design or estimation method to develop and execute a Transition Plan that describes how estimates produced by the new design will be incorporated into the existing, or legacy, time series.
Fully transitioning to a new survey design may require benchmarking and/or statistical calibration.
- Benchmarking allows different survey designs to be implemented side-by-side. The resulting estimates are examined, differences are measured, and potential drivers of differences are identified and evaluated.
- Calibration accounts for the sources of variation that may be contributing to consistent differences between old and new survey estimates. It can be used to convert historical estimates to the “currency” of the new survey design, thereby preserving the continuity of the historical time series.
The Marine Recreational Information Program’s Transition Team was established to facilitate the transition process. The steps of the transition process, as well as the recommended sections of a Transition Plan, are described in more detail in Procedural Directive 04-114-01 (PDF, 5 pages).
Certified Survey Designs
The Marine Recreational Information Program continues to prioritize the certification of existing survey designs, as well as the certification of and transition planning for new survey designs. Because this is a time- and resource-intensive process, it’s not possible to certify all of the methods currently in use at once. However, NOAA Fisheries can support the continued use of a non-certified, “legacy” survey design, as long as:
- The estimates produced by such a survey design have been used in a peer-reviewed application, such as a federal stock assessment.
- The need to continue using such a survey design has been expressed in an MRIP Regional Implementation Plan.
- A plan to certify such a survey design is in place and is being followed.
The program documents its ongoing progress toward certification and transition planning in its annual implementation plan updates. Additional documentation for the surveys listed below can be found in the MRIP Reports Database. These anticipated timelines are subject to change.
MRIP-Administered Survey Designs
|Access Point Angler Intercept Survey||NOAA Fisheries certified the APAIS design in 2012. In 2013, it was implemented from Maine through Mississippi; the transition to the APAIS along the Atlantic and Gulf coasts is complete. In 2023, it will be implemented in Hawaii; the transition to the APAIS in Hawaii is in progress.|
|Fishing Effort Survey||NOAA Fisheries certified the FES design in 2015. In 2018, it was implemented from Maine through Mississippi and in Hawaii. The transition to the FES is in progress.||
|For-Hire Survey||NOAA Fisheries certified the FHS design in 2023. This telephone survey is conducted along the Atlantic and Gulf coasts. A transition plan is in progress.||
|Large Pelagics Survey||While the Large Pelagics Telephone Survey design has been documented and peer reviewed, the certification of the full LPS design is not expected to occur until the redesign of the Large Pelagics Intercept Survey is complete.|
State and Regional Survey Designs
|LA Creel (Louisiana Department of Wildlife and Fisheries)||NOAA Fisheries certified the LA Creel design in 2017. A Transition Plan for LA Creel, Tails n’ Scales (MS), Snapper Check (AL), and the State Reef Fish Survey (FL) is under development.|
|Tails n’ Scales (Mississippi Department of Marine Resources)||NOAA Fisheries certified the Tails n’ Scales design in 2018. A Transition Plan for Tails n’ Scales, LA Creel (LA), Snapper Check (AL), and the State Reef Fish Survey (FL) is under development.|
|Snapper Check (Alabama Department of Conservation and Natural Resources)||NOAA Fisheries conditionally certified the Snapper Check design in 2018. A Transition Plan for Snapper Check, LA Creel (LA), Tails n’ Scales (MS), and the State Reef Fish Survey (FL) is under development.|
|State Reef Fish Survey (Florida Fish and Wildlife Conservation Commission)||NOAA Fisheries certified the State Reef Fish Survey design in 2018. (At the time of certification, SRFS was known as the Gulf Reef Fish Survey.) A Transition Plan for the State Reef Fish Survey, LA Creel (LA), Tails n’ Scales (MS), and Snapper Check (AL) is under development.|
|Ocean Sampling Program (Washington Department of Fish and Wildlife)||NOAA Fisheries certified WDFW's Ocean Sampling Program design in 2023.||
|Ocean Recreational Boat Survey (Oregon Department of Fish and Wildlife)||The certification of the ORBS design is in progress: Documentation has been received; the peer review is complete; and a response to the peer review is under development.|
|California Recreational Fisheries Survey (California Department of Fish and Wildlife)||NOAA Fisheries certified CDFW's CRFS Angler License Directory Online Survey design and Private/Rental Boat Recreational Fishing Surveys in 2023.|