Recreational Fishing Survey and Data Standards
These standards promote data quality, consistency, and comparability across the Marine Recreational Information Program’s national network of recreational fishing surveys.
NOAA Fisheries’ Recreational Fishing Survey and Data Standards guide the design, improvement, and quality of the information produced by the recreational fishing surveys that are administered or funded through the agency’s Marine Recreational Information Program. By providing a single set of guidelines for recreational fisheries data collection and estimation, the standards support the program’s commitment to transparency. By promoting data quality, consistency, and comparability across data collection programs, the standards facilitate the shared use of the statistics these programs produce. Ultimately, the standards will further ensure the integrity of our data collection efforts, the quality of our recreational fisheries statistics, and the strength of science-based management decisions.
These standards reflect best practices currently in place at the National Center for Health Statistics, U.S. Census Bureau, and other federal agencies, as well as statistical survey standards and guidelines published by the Office of Management and Budget.
All recreational fishing surveys that receive funding from the Marine Recreational Information Program are required to meet all of the standards below, with the exception of Standards 7.2.1, 22.214.171.124, and 126.96.36.199, which are specific to MRIP.
While survey administrators are responsible for developing the required survey documentation, state agencies may choose to work with their regional Fisheries Information Networks to prepare regional reports for submission to NOAA Fisheries.
While these standards were established in 2020, several are already in use: Standards 1-4 are part of NOAA Fisheries’ certification and transition policies and procedural directives; Standard 5 is part of cooperative agreements between the agency and its partners; and Standard 6.1 is part of the program’s regional implementation planning process.
The implementation of the remaining standards will be phased. To help our partners and data customers familiarize themselves with the standards and with changes to the way NOAA Fisheries will present its recreational fisheries statistics, the agency will:
- Deliver presentations to regional management groups (as requested).
- Distribute survey compliance checklists.
- Publish a Data User Handbook, to include guidance on downloading, exporting, querying, and analyzing the agency’s marine recreational fishing data.
- Hold Data User Workshop(s).
- Preview changes to the MRIP Query Tool.
Once the standards are fully implemented, previously published data will be updated. In other words, previously published estimates that do not meet the standards will not be provided as part of the agency’s marine recreational fisheries statistics. Analysts who choose to create non-standard estimates (e.g., wave-specific rather than cumulative estimates) may use the microdata and custom domain analysis programs available on the Recreational Fishing Data Downloads page and follow the guidance that will be provided in the MRIP Data User Handbook. However, we do not suggest or support the use of these statistics.
1. Survey Concepts and Justification
A written survey plan should, at minimum, describe:
- Survey goals and objectives, which should meet the key estimate and precision requirements described in Standard 7.2.2 and 188.8.131.52 respectively, and address priorities in the relevant Regional Implementation Plan.
- Legislation or executive orders that have mandated the data collection.
- Adherence to Office of Management and Budget (OMB) requirements, including OMB control number.
- Intended uses and users.
- Descriptions of changes for recurring surveys.
1.2: Paperwork Reduction Act Compliance
All information collections administered or sponsored by a federal agency must comply with the requirements of the Paperwork Reduction Act of 1995 (PRA). Survey administrators are responsible for developing supporting materials for OMB. If the survey administrator is not a federal agency, the federal agency sponsor is responsible for submitting these materials.
2. Survey Design
A technical document describing the survey objective(s) and basic data collection and estimation designs must be developed, reviewed, and certified (see Standard 5.1). The document must include the components described in Standards 2.1-2.4. For an example, see Survey Design and Statistical Methods for Estimation of Recreational Fisheries Catch and Effort (PDF, 98 pages).
A sampling plan must describe the survey’s target population, sample frame, sample unit, stratification, and methods of sample selection (e.g., simple random sampling, probability proportional to size sampling, justification for non-probability designs).
2.2: Data Collection
Descriptions of data collection procedures must include the anticipated frequency and timing of data collection, primary data collection mode(s), survey protocols, data elements, and survey instruments.
- 2.3.1: Sample Weights: A weighting plan must identify stages of weighting (e.g., base weights, nonresponse adjusted weights, calibrated/post-stratified weights) and include calculations and/or descriptions for how final weights are derived.
- 2.3.2: Point and Variance Estimate Calculation: Descriptions of estimation designs must include point and variance estimates. Estimates must be generated using a method that is statistically consistent with the survey’s sample design (e.g., unequal probabilities of selection, stratification, clustering, and the effects of nonresponse, post-stratification, and raking).
An evaluation plan must identify potential sources of non-sampling error, including nonresponse, undercoverage, and measurement. To the extent possible, this plan should evaluate potential impacts of non-sampling error on survey estimates and describe plans/results from bias analyses that attempt to mitigate and/or measure non-sampling error (e.g., nonresponse bias adjustments, post-stratification, calibration weighting).
3. Data Quality
3.1: Processing, Editing, and Quality Control
Survey documentation must describe data processing procedures, including methods to review and edit data to mitigate errors, as well as methods to compensate for item nonresponse. Survey data must identify actions taken during editing, and include both edited and unedited values.
3.2: Quality Assurance
Survey documentation must include a quality assurance plan for each phase of the survey process, to support performance monitoring and assessment. Quality assurance may include the training and supervision of data collectors, as well as the independent validation of interviews, where possible.
4. Transition Planning
Per NOAA Fisheries Policy Directive 04-114 (PDF, 4 pages), a transition plan must be prepared before the implementation of new or improved sampling or estimation designs that are likely to result in large deviations from historical estimates. Guidance is provided in Procedural Directive 04-114-01 (PDF, 5 pages).
5. Review Procedures
All survey designs are subject to the certification requirements set forth in NOAA Fisheries Policy Directive 04-114 (PDF, 4 pages). Certification requires a peer review of survey methods, as well as review and approval by the MRIP Executive Steering Committee. The required documentation is described in Standard 2: Survey Design and Standard 3: Data Quality.
5.2: Annual Reporting
An annual report for each survey must be submitted at the conclusion of each survey year. Annual reports must provide an overview of data collection procedures, including questionnaires and data collection schedules; the components described in Standards 5.2.1-5.2.7; and key survey estimates for each survey administration (e.g., reference wave) within the survey year. For an example, see the Fishing Effort Survey 2019 Annual Report (PDF, 48 pages).
- 5.2.1: Sample sizes: Overall, and by state/stratum.
- 5.2.2: Completed surveys: Overall, by state/stratum, and final disposition.
- 5.2.3: Response rates.
- 5.2.4: Summary of editing and corrective actions.
- 5.2.5: Changes in methodology.
- 5.2.6: Quality assurance measures to minimize sampling and non-sampling error.
- 5.2.7: Efforts to identify and evaluate alternative or modified designs that could improve the accuracy, efficiency, and timeliness of survey results.
5.3: Peer Review
Peer reviews of annual reports and information products (e.g., microdata and/or estimates) will be completed by Office of Science and Technology staff, who may request additional review from subject matter experts and/or statistical consultants. Peer reviews will comply with NOAA Fisheries Policy Directive 04-108 (PDF, 26 pages). Peer reviews will inform agency decisions to publish information products, and may identify the need to establish a process improvement plan, as described in Standard 6: Process Improvement.
6. Process Improvement
6.1: Process Improvement Plan
The ongoing evaluation of survey designs ensures methods address emerging needs and incorporate current best practices. Annual reports, as described in Standard 5.2, must discuss these efforts. Regional Implementation Plans should identify unmet needs. The MRIP Program Management Team will evaluate recommendations to implement or further explore revisions on a case-by-case basis, and may identify opportunities for additional research upon review of annual reports and Regional Implementation Plans.
6.2: Unplanned Modifications
In some instances, survey results (e.g., low response rates) or administrative challenges (e.g., budget shortfalls) may create or identify deficiencies that require unanticipated design modifications. The need for unplanned changes must be communicated to the MRIP Program Management Team as soon as possible.
7. Access and Information Management
Information products must include the data described in Standard 7.1 and the statistical elements described in Standard 7.2; these data and statistics must be associated with the attributes described in Standard 7.4. Files must be available in the formats described in Standard 7.3. Information products associated with data collections funded by NOAA Fisheries must meet the federal information management requirements described in Standard 7.5.
Survey data (i.e., observation data collected for each statistical unit).
- 7.1.1: Processed microdata (post-quality control and other edits) are published online with associated weights each survey year.
- 7.2.1: Cumulative Estimates: For each survey year, MRIP publishes cumulative estimates where estimates are available sub-annually, beginning with the first survey administration of the survey year.
- 7.2.2: Key Statistics:
- 184.108.40.206: Total (estimated or censused) finfish catch (landed and released) by year, state, fishing mode, area fished, and species.
- 220.127.116.11: Total (estimated or censused) finfish trips by year, state, mode, and area.
- 7.2.3: Measures of Precision for Estimates Posted Publicly: OMB has established Standards and Guidelines for Statistical Surveys that require agencies to identify criteria for determining when errors are too large for a survey estimate to be publicly released. The U.S. Census Bureau, also within the Department of Commerce, does not publicly release an estimate when its coefficient of variation exceeds 30 percent. Given the pulse nature and high variability of many recreational fisheries, MRIP has adopted a more liberal precision standard: MRIP presents a warning when the percent standard error (PSE) for an estimate exceeds 30 percent and will not publish an estimate when the PSE exceeds 50 percent.
- 18.104.22.168: Measures of Precision: All published estimates must include a point estimate and a measure of precision.
- 22.214.171.124: Estimates are not published if the PSE is greater than 50 percent (i.e., if the standard error represents more than 50 percent of the estimate).
- 126.96.36.199: Warnings are presented for estimates with PSEs between 30 and 50 percent. (Estimates with a PSE of 30 percent or greater are not considered sufficiently reliable for most purposes, and should be treated with caution.)
7.3: File Formats
- 7.3.1: Required: CSV
- 7.3.2: Recommended: SAS
7.4: Attribute Values and Formats
- 7.4.1: Geographic.
- 7.4.2: Temporal.
- 7.4.3: Fishing Modes.
- 7.4.4. Fishing Area.
- 7.4.5: Public Use Datasets.
7.5: Information Management
- 7.5.1: General
- 7.5.2: Access
- 7.5.3: Archive
- 7.5.4: Data Management
- 7.5.5: Documentation/Metadata
- 7.5.6: Citation
- 188.8.131.52: NOAA Data Citation Procedural Directive. Disclosure Avoidance: Prevent the unauthorized release of personally or business identifiable information (PII/BII).