Frequent Questions: Recreational Fishing Survey and Data Standards

Introduction

With input from our partners, including state agencies and regional fisheries information networks, NOAA Fisheries established recreational fishing survey and data standards in 2020 to promote data quality, consistency, and comparability across the recreational fishing surveys administered and funded through the agency’s Marine Recreational Information Program.

Reflecting federal guidelines and best practices for the publication of statistical information, the standards set clear criteria for sound recreational fishing survey management practices. By providing concise requirements for collecting data and producing and disseminating estimates, the standards ensure the integrity of data collection and the quality of recreational fisheries statistics, while also improving transparency.

When will the new standards be fully implemented?

To ensure a smooth transition, the standards have been rolled out in phases. The final phase, related to measures of precision for estimates posted publicly and cumulative estimates, is expected to be fully implemented in April 2023.

What will happen when NOAA Fisheries implements the measures of precision standard, and why was it developed?

Once this standard is in place, NOAA Fisheries will no longer publish estimates with a percent standard error (PSE) greater than 50 percent, and will present a warning when an estimate’s PSE exceeds 30 percent. The PSE is a measure of precision for our recreational catch estimates that indicates potential for error.

This precision threshold is representative of best practices currently in place at the National Center for Health Statistics, U.S. Census Bureau, and other federal agencies. It also reflects guidance from state and federal stock assessment scientists and managers from New England, the Mid-Atlantic, and the Southeast, who participated in a workshop to explore the effects of increasingly imprecise estimates on stock assessment results. Workshop participants agreed that estimates with PSEs greater than 40 percent should only be used with caution. The Atlantic Coastal Cooperative Statistics Program continues to set a goal of achieving PSE values of 20 to 30 percent.

By ensuring that MRIP estimates meet a defined level of precision, this particular standard will allow the agency to:

• Publish more reliable recreational fisheries statistics.
• Reduce the risks associated with the inappropriate interpretation of imprecise estimates.
• Direct resources toward improving statistical precision for data-limited fisheries, or those fisheries whose catch estimates tend to be imprecise because they are not often encountered through our sample surveys.

What if an estimate with a PSE of greater than 50 percent is needed for scientific or management purposes?

When the precision publication threshold prevents the release of an estimate that a stock assessment scientist or fisheries manager needs, Office of Science and Technology staff will work with the data customer to explore ways to improve statistical precision for the affected fisheries.

What will happen when NOAA Fisheries implements the cumulative estimates standard, and why was it developed?

Under the cumulative estimates standard, NOAA Fisheries will publish cumulative estimates every two months instead of publishing estimates specific to each two-month sampling “wave.” This will begin with the first survey administration of the survey year according to the following schedule:

By increasing the reference periods of the estimates we produce, we are effectively increasing the sample size of each estimate, which can increase statistical precision. In addition, this increased sample size will allow us to produce better estimates overall. It helps ensure that the interviews reflected in our estimates are representative of the entire population of anglers. The switch to cumulative estimates will also help us get PSEs below 50 percent faster and give data users access to the information they need sooner. They are a more natural fit for most fisheries management plans as well.

Will stock assessors still have access to wave-level estimates?

Analysts who choose to create non-standard estimates (for instance, wave-specific rather than cumulative estimates) may use the microdata and custom domain analysis programs available on the Recreational Fishing Data Downloads page and follow the guidance that will be provided in the MRIP Data User Handbook. However, MRIP does not suggest or support the use of these statistics.

If the statistics I need are not part of the agency's official estimates, will I still be able to access them?

Partners and the public will continue to have access to all of the microdata collected through our surveys (used to produce the estimates), as well as the tools needed to analyze these data. When the precision threshold prevents the release of an estimate that a stock assessment scientist or fisheries manager needs, the Office of Science and Technology will work with data customers to explore ways to improve statistical precision for the affected fisheries (e.g., by using a multi-year averaging technique). The precision threshold only applies to the publication of estimates (calculated statistical values).

Is it possible to preview the impact of the publication standard?

Yes. There is a preview of how catch and effort estimates will be published using the new data standards on the MRIP Query Tool. Under “Catch Data” or “Effort Data,” use the dropdown menu to go to “Preview of Data Standards.” Then click “Goto Query.”

Will these changes be retroactive?

Yes. Previously published data will be retroactively updated after these standards are put in place. Once these standards are fully implemented, wave-level estimates and imprecise estimates—defined as those with PSEs greater than 50 percent—will be removed from our published recreational fisheries statistics.

Which surveys must adhere to which standards?

All recreational fishing surveys that receive funding from the Marine Recreational Information Program are required to meet all of the standards, with the exception of Standards 7.2.1, 7.2.3.2, and 7.2.3.3.

How do certification requirements relate to the standards?

To earn certification, survey sponsors must provide documentation to support a technical review of survey methods. This documentation is the same documentation described in Standards 1, 2, and 3.

Are similar standards, guidelines, or best practices available elsewhere?

• Principles and Practices for a Federal Statistical Agency (National Academies of Sciences, Engineering, and Medicine)
• Standards and Guidelines for Statistical Surveys (Office of Management and Budget)
• Statistical Quality Standards (U.S. Census Bureau)
• Federal, National, and International Data Standards (U.S. Environmental Protection Agency)
• Data Standards (U.S. Geological Survey)
• Measuring and Reporting Sources of Error in Surveys (Office of Management and Budget)
• Data Presentation Standards for Proportions (National Center for Health Statistics)
• Criteria for Data Suppression (Centers for Disease Control and Prevention)
• American Community Survey Data Suppression (U.S. Census Bureau)
• Behavioral Risk Factor Surveillance System Data User Guide (Centers for Disease Control and Prevention)
• American Community Survey Data User Notes (U.S. Census Bureau)

Implementation

What is NOAA Fisheries doing to help its partners transition to the standards?

To help our partners and data customers familiarize themselves with the standards and with changes to the way NOAA Fisheries will present its recreational fisheries statistics, the agency has:

• Delivered presentations to regional Fisheries Information Networks.
• Published the MRIP Data User Handbook, which contains detailed information about downloading, exporting, querying, and analyzing the agency’s marine recreational fishing data.
• Published new queries to provide a preview of our switch to cumulative (rather than wave-specific) estimates (see Standard 7.2.1) and our adoption of a precision threshold (see Standard 7.2.3.2).
• Held a Data User Seminar Series to provide stock assessors and fisheries analysts with best practices for accessing, analyzing, and using recreational fishing data.

The agency will also develop a database for the documentation survey administrators must submit to meet the standards.
If additional support is needed, partners and data customers are encouraged to contact members of the Marine Recreational Information Program’s Program Management Team or identify needs in their next Regional Implementation Plan.

Will National Saltwater Angler Registry Memoranda of Agreement between NOAA Fisheries and States need to be updated as part of the transition to the standards?

The preferred approach would involve cooperatively working with state agencies to achieve the standards.

Documentation

Several standards require survey administrators to develop documentation. Recreational Fishing Survey and Data Standards Documentation Guidance (PDF, 7 pages) explains how these requirements should be met and answers the questions below in more detail.

Who is responsible for preparing documentation to meet the standards?

Survey administrators are responsible for developing required survey documentation. In some cases, multiple entities may be responsible for different aspects of survey administration. In those cases, different pieces of documentation may be developed by different entities. Recreational Fishing Survey and Data Standards Documentation Guidance (PDF, 7 pages) describes how programs can meet these requirements and, for MRIP-funded surveys, lists the entities that are responsible for preparing and submitting annual reports.

What is an example of sufficient documentation?

While there is some flexibility in how documentation requirements can be met, Recreational Fishing Survey and Data Standards Documentation Guidance (PDF, 7 pages) proposes two documentation models for consideration.

• The first follows the model of the MRIP Fishing Effort Survey, which meets applicable documentation requirements through a comprehensive survey design document, a transition plan, and an annual report.

• The second follows the model of the MRIP Access Point Angler Intercept Survey, which is unique in the fact that multiple entities are responsible for different aspects of the data collection program. In this case, the lead survey administrator meets some of the applicable documentation requirements through a survey plan (PDF, 1 page) and a comprehensive survey design document. Partners meet the remaining requirements through an annual report.

When is documentation due, and how often should it be revised?

Documentation related to Standards 1.1, 2.1, 3.1, 3.2, and (if applicable) 4 should be submitted to NOAA Fisheries as soon as it is available. In most cases, these documents will only need to be produced once, but revised as survey methods change. Annual reports, on the other hand, should be completed following the receipt of final estimates for a calendar year, and generally be submitted to NOAA Fisheries by June 30 of the following year.

How should documentation be submitted?

All documentation must be uploaded to the NOAA Fisheries Research Publication Tracking System (RPTS). Partners with NOAA email accounts (i.e., @noaa.gov) may self-register to access RPTS. Non-NOAA users and anyone needing support should contact the NOAA Fisheries RPTS Administrator or Lauren Dolinger Few.

Where will documentation be stored?

All documentation will be publicly available in the MRIP Reports Database.

What if documentation is already available elsewhere?

To achieve transparency, documentation should be organized to clearly meet the purpose of the standards. Existing documents may be submitted as long as the requirements of the standards are met and clearly identified.

Standard 7.2.3: Measures of Precision for Estimates Posted Publicly

How will the precision threshold improve the quality of recreational fisheries statistics?

The precision threshold described in Standard 7.2.3 is an important part of improving the quality of our recreational fisheries statistics. Ultimately, the precision threshold will:

• Provide scientists and managers with more reliable catch estimates.
• Reduce the risks that are associated with the inappropriate interpretation of estimates that are imprecise.
• Indicate when data limited fisheries could benefit from additional data collection support and where alternatives in survey design, sample allocation, or management may be necessary.

Is NOAA Fisheries' precision threshold consistent with the practices of other federal agencies?

The precision threshold reflects practices currently in place at the National Center for Health Statistics, U.S. Census Bureau, and other federal statistical agencies. The U.S. Census Bureau, for instance, does not publicly release estimates whose coefficients of variation exceed 30%. Our precision threshold is more liberal: we will present a warning when an estimate’s percent standard error exceeds 30%, and will not publish an estimate when its percent standard error exceeds 50%.

What is the Marine Recreational Information Program doing to produce more precise estimates?

In FY20, the program distributed $3 million in Modern Fish Act investment funds to its Atlantic, Gulf, and Pacific Regional Implementation Teams to increase sampling levels and improve the precision of catch estimates. The program is also exploring ways to produce more precise estimates for rare-event species.

More Information

• NOAA Fisheries Establishes Recreational Fishing Survey and Data Standards 
• Recreational Fishing Survey and Data Standards
• Survey and Data Standards Documentation Guidance (PDF, 7 pages)
• MRIP Data User Handbook (PDF, 51 pages)
• MRIP Data Downloads
• MRIP Query Tool