Overview
Research Track assessments provide detailed evaluation and analysis of research topics that are related to stock assessments. They carry out analysis that are peer reviewed by an independent research panel. During the fall of 2020, there will be a research track assessment to evaluate index based methods and control rules. This work will be done by a working group.
This working group will use standard simulation approaches to explore a variety of index-based assessment approaches. It will consider:
- Which index-based methods perform best for which harvest control rules.
- What types of situations are or are not suited to a particular method or control rule.
- What guidelines are appropriate for setting biological reference points.
It will not address the challenges associated with missing or partial survey data or logistical issues.
Working Group Members
Chris Legault, Chair
Rich Bell
Jamie Cournane
Jon Deroba
Gavin Fay
Andy Jones
Tim Miller
Brandon Muffley
John Weidenmann
Understanding Retrospective Patterns
Recently, the use of index-based assessments has increased. Analytical assessments for several stocks no longer accurately represent the stocks’ population dynamics. In many cases, this has been evident by strong retrospective patterns in assessment projections and outcomes.
What Are Retrospective Patterns?
Retrospective patterns are systematic patterns in an analytical assessment where trends in recent model estimates are either higher or lower than previous trends. These are identified by comparing model estimates of important quantities, like population biomass, using different time frames. If the model estimates from the full time series diverge from model estimates that have had recent years of data removed, and there is a systematic pattern, we call this a retrospective pattern.
A strong retrospective pattern indicates inconsistency in the data and model. This prevents using standard simulation testing approaches to audit the assessment, and has increased our use of index-based assessments.
Terms of Reference
- Develop methods to create data that if assessed with standard age-based approaches (e.g., VPA or ASAP) could exhibit a strong retrospective pattern.
- Identify a number of index-based methods and a range of harvest control rules for use in closed-loop simulation, using index-based data resulting from ToR 1.
- Identify metrics from the index-based assessment results that could be used in evaluations of trade-offs in performance among harvest control rules and index-based methods.
- Evaluate the combinations of index-based methods and control rules using the metrics in ToR 3 to determine candidates for consideration by the Councils or other management authorities.
- Provide guidance on specific situations that are and are not well-suited for a particular control rule or index-based method identified in ToR 4.
- Create guidelines for setting biological reference points for index-based stocks.
Meetings
- Peer Review Meeting
- November Meetings
- October Meetings
- September Meetings
- August Meetings
- July Meetings
- June Meetings
- May Meetings
- April Meetings
- Kick-off Meeting
Petition for Working Group Membership
To become a member of the working group,complete the questionnaire. Indicate the research track working group name on the application. Applications should be submitted by February 7, 2020. You can email them to michele.traver@noaa.gov or send them to: Michele Traver, c/o Northeast Fisheries Science Center, 166 Water Street, Woods Hole MA 02543.
The questionnaire is an Adobe PDF. If you would prefer to receive the document in Microsoft Word, or if you have any questions about the process, please contact Michele either by email or at: (508) 495-2195. Upon submission of your questionnaire, you will receive an email confirmation.
- Questionnaire instructions (PDF; 2 pages): Download File
- Questionnaire (PDF; 2 pages): Download File
Questions may be directed to Michele Traver at (508) 495-2195.
Timeline
- Questionnaire available to public: January 17, 2020
- Responses due back: February 7, 2020
- Short list selected for the working group by: February 14, 2020 (target date)
- All questionnaires and short list delivered to NRCC Decision Committee: February 21, 2020 (target date)
- NRCC decisions due to NEFSC: February 28, 2020 (target date)
- Public announcement of decisions: March 6, 2020 (target date)
