Frequent Questions: Recreational Fishing Survey and Data Standards

Introduction

With input from our partners, including state agencies and regional fisheries information networks, NOAA Fisheries established recreational fishing survey and data standards in 2020 to promote data quality, consistency, and comparability across the recreational fishing surveys administered and funded through the agency’s Marine Recreational Information Program.

Reflecting federal guidelines and best practices for the publication of statistical information, the standards set clear criteria for sound recreational fishing survey management practices. By providing concise requirements for collecting data and producing and disseminating estimates, the standards ensure the integrity of data collection and the quality of recreational fisheries statistics, while also improving transparency.

What are the NOAA Fisheries recreational fishing survey and data standards?

The standards (initiated in late 2020 through phased implementation) set clear criteria to guide the design, improvement, and quality of data produced by the agency’s recreational fishing surveys. The standards also align with requirements and best practices of other federal agencies that produce statistics for decision-making. They provide concise requirements for data collection and distribution, thereby promoting data consistency across the national network of recreational fishing surveys. The standards help reduce ambiguity and potential misinterpretation of recreational fishing data.

These standards serve as a single set of guidelines on seven aspects of recreational fisheries data collection and estimation. These focus areas include:

• Survey Concepts and Justification: Surveys must address identified priorities, produce key statistics, and include measures of precision. Written survey plans must describe: Survey goals, legislative mandates, adherence to Office of Management and Budget and (if applicable) Paperwork Reduction Act guidelines, and intended uses and users.
• Survey Design: Documentation must describe sampling plans (who the survey will be administered to), evaluation plans, and data collection and estimation designs.
• Data Quality: Documentation must describe procedures for data processing, methods to make up for item nonresponse, actions taken during data editing, and the quality assurance plans that are in place for each phase of the survey process.
• Transition Planning: Transition plans must be prepared before new or improved data collection or estimation designs are implemented if the designs are likely to result in large or meaningful deviations from historical estimates.
• Review Procedures: Data collection and estimation designs are subject to existing certification requirements. Annual reports must be submitted at the end of each survey year, and peer reviews of both annual reports and information products will be completed by the NOAA Fisheries Office of Science and Technology.
• Process Improvement: The ongoing evaluation of survey designs should address emerging needs and incorporate best practices. Recommended revisions and unanticipated modifications must be documented, reported, and evaluated.
• Access and Information Management: Survey data, microdata, and measures of precision must be available online, and data collections funded by NOAA Fisheries are subject to existing information management requirements. NOAA Fisheries Marine Recreational Information Program presents a warning when the percent standard error for an estimate exceeds 30% and recommends it should be used with caution; and does not support the use of highly imprecise estimates when the percent standard error (PSE) exceeds 50%. For highly imprecise estimates, MRIP recommends aggregation (e.g., across states, geographic regions, fishing modes, and/or years). Aggregating data (in this case, combining data from all prior waves in the same calendar or fishing year) is a common statistical approach to increase sample sizes over time and smooth spikes or anomalies in the data. The goal of increasing sample sizes is for the catch and effort estimates to become more precise throughout the year. 

These standards apply to which surveys?

All data collection programs administered by NOAA Fisheries’ Marine Recreational Information Program will adhere to these standards, including the Fishing Effort Survey (mailed to households in coastal states); Access Point Angler Intercept Survey (conducted via in-person interviews); For-Hire Survey (telephone survey based on the for-hire vessel directory); and the Large Pelagics Survey (conducted via telephone and in-person) for highly migratory species. 

The majority of these standards are relevant to state programs certified or sponsored (funded) by MRIP. The phased implementation of these standards allowed for state and regional fisheries information network partners to incorporate applicable provisions of the standards into MRIP-supported state surveys.

What is the precision standard?

As part of the government’s guidelines for statistical programs, the Office of Management and Budget requires federal agencies to establish their own criteria for determining when an estimate is too unreliable or imprecise to publicly release (precision standard).

NOAA Fisheries Marine Recreational Information Program presents a warning when the percent standard error for an estimate exceeds 30% and recommends it should be used with caution; and does not support the use of highly imprecise estimates when the percent standard error (PSE) exceeds 50%. The higher the PSE, the larger the margin of error or uncertainty with the estimate.

Our precision standard was developed through collaborative feedback from partners who explored effects of imprecise estimates on stock assessment results. Partners determined estimates above 40 PSE should be used with caution. The U.S. Census Bureau does not provide estimates with a PSE above 30. The Atlantic Coastal Cooperative Statistics Program continues to set a goal of achieving PSEs below 30.

The precision standard creates flexibility for data users to pursue alternative analysis options that use more precise estimates for data-limited fisheries. It also reduces the risk of using imprecise catch and effort estimates to inform fisheries management decisions. 

Was there an opportunity to provide input on the survey and data standards prior to implementation? 

In 2019, prior to the standards being finalized, we solicited detailed feedback from internal and external parties. We leveraged our partnerships with interstate fisheries commissions and fisheries information networks to connect with state partners and obtain feedback.

We used feedback we received to make changes to our data queries (i.e. adding several different fishing-year/season catch and effort estimation options), timeline for implementation (phased to provide adequate adaptation time for data users), and support to our partners. We developed several tools, such as a data-user handbook, and provided seminars for customers and the public prior to and during the implementation of the standards.

We remain committed to providing our partners and data users with the support necessary to help inform and improve fisheries management. We will work with our data customers to explore ways to improve estimates for data-limited fisheries (e.g., by using multi-year averaging techniques, increasing sample sizes).

How do certification requirements relate to the standards?

To earn certification, survey sponsors must provide documentation to support a technical review of survey methods. This documentation is the same documentation described in Standards 1, 2, and 3.

Are similar standards, guidelines, or best practices available elsewhere?

• Principles and Practices for a Federal Statistical Agency (National Academies of Sciences, Engineering, and Medicine)
• Standards and Guidelines for Statistical Surveys (Office of Management and Budget)
• Statistical Quality Standards (U.S. Census Bureau)
• Federal, National, and International Data Standards (U.S. Environmental Protection Agency)
• Data Standards (U.S. Geological Survey)
• Measuring and Reporting Sources of Error in Surveys (Office of Management and Budget)
• Data Presentation Standards for Proportions (National Center for Health Statistics)
• Criteria for Data Suppression (Centers for Disease Control and Prevention)
• American Community Survey Data Suppression (U.S. Census Bureau)
• Behavioral Risk Factor Surveillance System Data User Guide (Centers for Disease Control and Prevention)
• American Community Survey Data User Notes (U.S. Census Bureau)

Will National Saltwater Angler Registry Memoranda of Agreement between NOAA Fisheries and States need to be updated as part of the transition to the standards?

The preferred approach would involve cooperatively working with state agencies to achieve the standards.

Documentation

Several standards require survey administrators to develop documentation. Recreational Fishing Survey and Data Standards Documentation Guidance (PDF, 7 pages) explains how these requirements should be met and answers the questions below in more detail.

Who is responsible for preparing documentation to meet the standards?

Survey administrators are responsible for developing required survey documentation. In some cases, multiple entities may be responsible for different aspects of survey administration. In those cases, different pieces of documentation may be developed by different entities. Recreational Fishing Survey and Data Standards Documentation Guidance (PDF, 7 pages) describes how programs can meet these requirements and, for MRIP-funded surveys, lists the entities that are responsible for preparing and submitting annual reports.

What is an example of sufficient documentation?

While there is some flexibility in how documentation requirements can be met, Recreational Fishing Survey and Data Standards Documentation Guidance (PDF, 7 pages) proposes two documentation models for consideration.

• The first follows the model of the MRIP Fishing Effort Survey, which meets applicable documentation requirements through a comprehensive survey design document, a transition plan, and an annual report.

• The second follows the model of the MRIP Access Point Angler Intercept Survey, which is unique in the fact that multiple entities are responsible for different aspects of the data collection program. In this case, the lead survey administrator meets some of the applicable documentation requirements through a survey plan (PDF, 1 page) and a comprehensive survey design document. Partners meet the remaining requirements through an annual report.

When is documentation due, and how often should it be revised?

Documentation related to Standards 1.1, 2.1, 3.1, 3.2, and (if applicable) 4 should be submitted to NOAA Fisheries as soon as it is available. In most cases, these documents will only need to be produced once, but revised as survey methods change. Annual reports, on the other hand, should be completed following the receipt of final estimates for a calendar year, and generally be submitted to NOAA Fisheries by June 30 of the following year.

How should documentation be submitted?

All documentation must be uploaded to the NOAA Fisheries Research Publication Tracking System (RPTS). Partners with NOAA email accounts (i.e., @noaa.gov) may self-register to access RPTS. Non-NOAA users and anyone needing support should contact the NOAA Fisheries RPTS Administrator or Lauren Dolinger Few.

Where will documentation be stored?

All documentation will be publicly available in the MRIP Reports Database.

What if documentation is already available elsewhere?

To achieve transparency, documentation should be organized to clearly meet the purpose of the standards. Existing documents may be submitted as long as the requirements of the standards are met and clearly identified.

More Information

• Recreational Fishing Survey and Data Standards
• Survey and Data Standards Documentation Guidance (PDF, 7 pages)
• MRIP Data User Handbook (PDF, 49 pages)
• MRIP Data Downloads
• MRIP Query Tool