Effort Survey Improvements
In 2018, the Marine Recreational Information Program transitioned to a new, mail-based survey to collect effort information from shore and private boat recreational anglers in Hawaii and on the Atlantic and Gulf coasts.
In its 2006 review of the methods we used to collect recreational fishing data and report recreational fishing catch, the National Research Council recommended fundamental changes to our methods of data collection and acknowledged the limitations of collecting effort information through random digit dialing. In response, the Marine Recreational Information Program (MRIP) began to explore potential improvements to the way we understand and estimate the number of trips taken by shore and private boat anglers in Hawaii and on the Atlantic and Gulf coasts. In 2018, we adopted a new Fishing Effort Survey (FES).
Why We Made the Change
When the Marine Recreational Fisheries Statistics Survey—the precursor to MRIP—was established in 1979, the Coastal Household Telephone Survey (CHTS) was the method we used to collect data about angler effort. The CHTS used random-digit dialing to reach residential households in Hawaii and in counties on the Atlantic and Gulf coasts. While random-digit dialing was a standard sampling methodology for conducting household surveys, there were known limitations to this approach.
- Random-digit dialing was an inefficient method of sampling anglers, as it often placed calls to non-angler households.
- A telephone survey approach was susceptible to measurement error, or the risk of obtaining inaccurate or incomplete answers from respondents.
- As landlines were abandoned for wireless numbers and caller ID became more common, random-digit dialing began to reach fewer and fewer potential anglers.
- As response rates to the entire spectrum of telephone surveys declined, random-digit dialing became increasingly susceptible to the risk of anglers not responding to our calls.
In 2008, we launched a series of pilot studies to determine how to reduce potential biases in our effort survey.
Between 2008 and 2015, we conducted six pilot studies to identify a more accurate and efficient way to estimate recreational fishing effort in Hawaii and on the Atlantic and Gulf coasts.
- Establish an improved method of collecting effort information from anglers.
- Determine how fishing license and registration information can support our fishing surveys.
- Determine how to maximize response rates.
We tested four methods of collecting effort information from shore and private boat recreational anglers:
- A telephone survey that used fishing license information rather than random-digit dialing as its sampling frame.
- A telephone survey that used fishing license information and random-digit dialing as its sampling frame.
- A mixed-mode telephone and mail survey.
- A mail survey that used fishing license information and a mailing address database from the U.S. Postal Service as its sampling frame.
- A database of mailing addresses provides more representative samples than random-digit dialing.
- Mail surveys produce higher response rates than telephone surveys.
- Mail surveys produce timely effort estimates.
- Mail surveys can be more efficient and less expensive when fishing license and registration lists are used to identify potential anglers from samples drawn from a mailing address database. (Too many anglers are unlicensed for licenses and registrations to be our sole source of angler contact information.)
The fact that mail surveys allowed us to more accurately sample anglers—where random-digit dialing underrepresented this group—had an impact on the effort estimates these surveys generated: effort estimates generated through mail surveys were two to six times higher than effort estimates generated through the CHTS.
In 2014, the final report from our work to compare the CHTS to a mail survey design underwent rigorous external peer review. In 2015, this survey design was certified as a suitable CHTS replacement. A transition plan was needed to support our shift to this new survey.
In 2014, a cross-disciplinary Transition Team made up of state partners, scientists, stock assessors, and managers oversaw the development of a plan (PDF, 34 pages) to support our transition from the CHTS to the FES. This plan ensured potential impacts to science and management processes were thought-out and accounted for before we transitioned to a new survey design.
In 2014, the MRIP Transition Team developed a plan to support our transition from the CHTS to the FES.
In 2015, 2016, and 2017, the FES was conducted alongside the CHTS to estimate shore and private boat recreational fishing effort in Hawaii and on the Atlantic and Gulf coasts.
The FES Transition Progress Report (PDF, 10 pages) describes the results from the first full year of this benchmarking period.
Developing a Calibration Model
Calibration is a critical step in transitioning to a new survey design. When a survey design is altered, the resulting estimates can begin to show consistent differences from those that were produced before changes in sampling methods were made. Because our work to foster healthy, productive, and sustainable marine fisheries depends on an historical time series of fishing statistics, the historical effort estimates produced by the CHTS needed to be converted into what they would have been if the new, mail survey design had been in place all along. Calibrating historical data is a technical process that places legacy estimates and new estimates into the same “currency,” allowing “apples to apples” comparisons between the two.
Between 2016 and 2017, we worked with expert consultants from Colorado State University to measure and evaluate consistent differences between the estimates produced from the CHTS and the estimates produced from the FES. This allowed us to determine whether these differences could be explained by possible sources of bias in the CHTS design. This information was used to develop a calibration model for re-estimating historical recreational fishing statistics.
Possible Effects of Calibration Scenarios on Stock Assessments Planned for the MRIP FES Transition (PDF, 29 pages) describes the possible impacts a suite of calibration scenarios could have on stock assessments.
Discontinuing the CHTS and Implementing the FES
The CHTS was discontinued on December 31, 2017. As of January 1, 2018, all effort estimates will be generated through the FES.
Re-estimating Historical Recreational Fishing Statistics
After our calibration model for re-estimating historical effort estimates was peer reviewed and approved, we converted historical effort estimates dating back to 1981 to the FES “currency.” A similar peer-reviewed process was used to adjust historical catch estimates produced by the Access Point Angler Intercept Survey (APAIS) following a 2013 transition to an improved sampling design.
Incorporating Revised Statistics into Stock Assessments and Management Actions
In 2018, federal stock assessments will begin to incorporate recreational fishing statistics that have been revised following our transition to the FES. It will take several years to complete these assessments, and the results will inform management actions and decision-making related to stock status, annual catch limits, and allocation. The order in which these stock assessments will be completed was determined with input and guidance from the Transition Team.
The effort estimates derived from the FES are much higher than the effort estimates derived from the CHTS. But higher effort estimates do not necessarily mean there are more people fishing or fewer fish to catch. In fact, our research indicates this increase in effort estimates is due not to a sudden increase in fishing, but to the fact that the FES is more effective at estimating fishing effort.
- The FES allows us to reach a more representative sample of anglers. Over the past 15 years, the percentage of adults living in wireless-only households—which the CHTS was unable to reach—has increased significantly across the United States. Including wireless-only households in our sample frame allows us to reach households that are more reflective of each state's population. Households that primarily use landline telephones, for example, report older residents, fewer children, and smaller household sizes. These households are also less likely to report fishing activity.
- The FES improves the likelihood of our questions getting into the right hands. By surveying the residents who are most familiar with their household's fishing activity, we can minimize the "gatekeeper effect."
- The FES gives respondents more time to provide more accurate and more complete answers about their fishing activity.
- The FES is designed as a weather and outdoor activity survey. By asking questions about topics other than fishing and including a $2 incentive, we can improve the number of responses we receive and reduce the risk of nonresponse bias.
Conducting the FES alongside the CHTS allowed us to evaluate differences between the estimates each survey produced and develop a model to convert historical fishing estimates into the new effort survey’s “currency.” Calibration showed us just how much higher angler effort was than the CHTS indicated. The fact that we have underestimated angler effort in the past indicates we may have underestimated historical fish abundance, too.