Effort Survey Improvements
In 2018, we transitioned to a new, mail-based survey to collect effort information from shore and private boat recreational anglers in Hawaii and on the Atlantic and Gulf coasts, and calibrated historical effort estimates to be comparable with this new survey.
In its 2006 review of the methods we used to collect recreational fishing data and report recreational fishing catch, the National Research Council recommended fundamental changes to our data collection techniques. In response, the Marine Recreational Information Program (MRIP) began to explore potential improvements to the way we understand and estimate the number of trips taken by shore and private boat anglers in Hawaii and on the Atlantic and Gulf coasts. In 2018, we adopted a new Fishing Effort Survey (FES).
Why We Made the Change
When the Marine Recreational Fisheries Statistics Survey—the precursor to MRIP—was established in 1979, the Coastal Household Telephone Survey (CHTS) was the method we used to collect data about angler effort. The CHTS used random-digit dialing to reach residential households in Hawaii and in counties on the Atlantic and Gulf coasts. While random-digit dialing was a standard sampling methodology for conducting household surveys, there were shortcomings to this approach.
- Random-digit dialing was an inefficient method of sampling anglers, as it often placed calls to non-angler households.
- As landlines were abandoned for wireless numbers, random-digit dialing reached fewer and fewer potential anglers.
- As response rates to telephone surveys declined, random-digit dialing became increasingly susceptible to the risk of anglers not responding to our calls.
- A telephone-based survey approach was susceptible to measurement error, or the risk of obtaining inaccurate or incomplete answers from respondents.
In 2008, we launched a series of pilot studies to determine how to reduce potential biases in our effort survey.
Between 2008 and 2015, we conducted six pilot studies to identify a more accurate and efficient way to estimate recreational fishing effort in Hawaii and on the Atlantic and Gulf coasts.
- Establish an improved method of collecting effort information from anglers.
- Determine how fishing license and registration information can support our fishing surveys.
- Determine how to maximize response rates.
We tested four methods of collecting effort information from shore and private boat recreational anglers:
- A telephone-based survey that used fishing license information rather than random-digit dialing as its sampling frame.
- A telephone-based survey that used fishing license information and random-digit dialing as its sampling frame.
- A mixed-mode telephone and mail-based survey.
- A mail-based survey that used fishing license information and a mailing address database from the U.S. Postal Service as its sampling frame.
- Address samples provide more representative samples than random-digit dialing.
- Mail-based surveys produce higher response rates than telephone-based surveys.
- Mail-based surveys produce timely effort estimates.
- Mail-based surveys can be more efficient and less expensive when fishing license and registration lists are used to identify potential anglers from samples drawn from a mailing address database. (Too many anglers are unlicensed for licenses and registrations to be our sole source of angler contact information.)
The fact that mail-based surveys allowed us to more accurately sample anglers—where random-digit dialing underrepresented this group—had an impact on the effort estimates these surveys generated: effort estimates generated through mail-based surveys were two to six times higher than effort estimates generated through the CHTS.
In 2014, the final report from our work to compare the CHTS to a mail-based survey design underwent external peer review. In 2015, this survey design was certified as a suitable CHTS replacement. A transition plan was needed to support our shift to this new survey.
In 2014, a cross-disciplinary Transition Team made up of state partners, scientists, stock assessors, and managers oversaw the development of a plan (PDF, 34 pages) to support our transition from the CHTS to the FES. This plan ensured potential impacts to science and management processes were thought-out and accounted for before we transitioned to a new sampling design.
In 2014, the MRIP Transition Team developed a plan to support our transition from the CHTS to the FES.
In 2015, 2016, and 2017, the FES was conducted alongside the telephone-based CHTS to estimate shore and private boat recreational fishing effort in Hawaii and on the Atlantic and Gulf coasts.
Developing a Calibration Model
Calibration is a critical step in the transition to a new survey design. When a survey design is altered, the resulting estimates can begin to show consistent differences between those that were produced before changes in sampling methods were made. Because our work to foster healthy, productive, and sustainable marine fisheries depends on an historical time series of fishing statistics, the historical effort estimates produced by the CHTS needed to be converted into what they would have been if the new, mail-based survey design had been in place all along. Calibrating historical data is a technical process that places legacy estimates and new estimates into the same “currency,” allowing “apples to apples” comparisons between the two.
Between 2016 and 2017, we worked with expert consultants from Colorado State University to measure and evaluate consistent differences between the estimates produced from the CHTS and the estimates produced from the FES. This allowed us to determine whether these differences could be explained by possible sources of bias in the CHTS design. This information was used to develop a calibration model for re-estimating historical recreational fishing statistics.
Discontinuing the CHTS and Implementing the FES
The CHTS was discontinued on December 31, 2017. As of January 1, 2018, all effort estimates will be generated through the FES.
Re-estimating Historical Recreational Fishing Statistics
After our calibration model for re-estimating historical effort estimates was peer reviewed and approved, we converted historical effort estimates dating back to 1981 to the FES “currency.” A similar peer-reviewed process was used to adjust historical catch estimates produced by the Access Point Angler Intercept Survey (APAIS) following a 2013 transition to an improved sampling design.
Incorporating Revised Statistics into Stock Assessments and Management Actions
In 2018, federal stock assessments will begin to incorporate recreational fishing statistics that have been revised following our transition to the FES. It will take several years to complete these assessments, and the results will inform management actions and decision-making related to stock status, annual catch limits, and allocation. The order in which these stock assessments will be completed was determined with input and guidance from the Transition Team.
The effort estimates derived from the FES are much higher than the effort estimates derived from the CHTS. But higher effort estimates do not necessarily mean there are more people fishing or fewer fish to catch. In fact, our research indicates this increase in effort estimates is due not to a sudden increase in fishing, but to the fact that the FES is more effective at estimating fishing effort.
- The FES allows us to reach a more representative sample of anglers.
- The FES improves the likelihood of our questions getting into the right hands.
- The FES gives respondents more time to provide more accurate and more complete answers about their fishing activity.
- The FES generates three times the response rate of the CHTS.
Conducting the FES alongside the CHTS allowed us to evaluate differences between the estimates each survey produced and develop a model to convert historical fishing estimates into the new effort survey’s “currency.” Calibration showed us just how much higher angler effort was than the CHTS indicated. The fact that we have underestimated angler effort in the past indicates we may have underestimated historical fish abundance, too.