Independent Peer Review of NOAA Fisheries’ Puget Sound Nearshore Calculator

Background

NOAA initiated an independent peer review of its Puget Sound Nearshore Calculator in 2023. The review was performed through NOAA’s Center of Independent Experts (CIE). NOAA’s CIE provides external, independent expert reviews of NOAA’s science. These independent expert reviews help ensure the use of best scientific information available as required by the Endangered Species Act and the Magnuson-Stevens Fishery Conservation and Management Act. The CIE selects national and international reviewers that meet high peer review standards such as high subject matter qualifications, independence from the science tools reviewed, and strict conflict of interest standards. Third party contract agreements with the CIE are administered by NOAA’s Office of Science and Technology. Reviewers must address all the review’s questions. The content of the reviewer reports is not open to changes by NOAA.

Five independent reviewers reviewed the Nearshore Calculator answering three sets of questions about whether (1) the underlying relationships that the Nearshore Calculator is built upon (e.g., duration of aquatic access, functional pathways, indicators/metrics) are well-founded and analytically sound; (2) the Nearshore Calculator systematically and appropriately incorporates and interprets the highest priority and best available scientific information given the stated goals and objectives; and (3) the Nearshore Calculator generates reasonable and well-supported quantifications of the impacts (positive, negative, neutral) to nearshore habitats and salmon. The reports can be found below:

• Dr. Mike Bradford peer review (PDF, 29 pages)
• Dr. Steven J. Cooke peer review (PDF, 22 pages)
• Dr. Eva C. Enders peer review (PDF, 17 pages)
• Dr. Robert Lennox peer review (PDF, 24 pages)
• Dr. Steven F. Railsback peer review (PDF, 23 pages)

Executive Summaries’ Highlights

Dr. Mike Bradford

Adjunct Professor, School of Environment and Resource Management, Simon Fraser University, Vancouver, BC, Canada

"This review was extensive, and I concluded for the most part, the knowledge inputs to the Calculator was based on the best available science. Analyses of additional habitat types, and offset measures, could be added to the calculator as part of ongoing improvements to the tool….in the future the developers of the Calculator should consider explicitly including the risks of failure of offsetting in their equivalency determinations."

Dr. Mike Bradford peer review (PDF, 29 pages)

Dr. Steven J. Cooke

Professor, Biology Department and Institute of Environmental Science, Carleton University, Ottawa, ON, Canada

"I remain in awe at the extent to which the Calculator has been thoughtfully developed drawing upon published literature, grey literature, and expert knowledge. The calculator itself is robust but the potential role of the calculator is overstated in that it is implied that use of the calculator will inherently “ensure” no net loss of valuable habitat. Climate change and cumulative effects are acknowledged in the report but there is need to further refine the tool to incorporate both uncertainty in inputs as well as recognize that off-sets that work for today may not be relevant for tomorrow. That does not invalidate the tool but is a cautionary reality."

Dr. Steven J. Cooke peer review (PDF, 22 pages)

Dr. Eva C. Enders

Associate Professor, Institut national de la recherche scientifique (INRS), Centre Eau, Terre et Environnement (ETE), Québec, QC, Canada

"To my knowledge the tool is based on the most current and reliable scientific information available on this subject and I found the Nearshore Calculator a very well-developed tool with a very detailed description of the model development and uncertainties as well as transparency on the model parameterization. However, there is always opportunity for model refinements, and I would recommend including the described uncertainty around the model parameter and interactions between environmental stressors into the mathematical analysis."

Dr. Eva C. Enders peer review (PDF, 17 pages)

Dr. Robert Lennox

Associate Professor, Dalhousie University, Halifax, Nova Scotia, Canada and Ocean Tracking Network, Halifax, NS, Canada

"There will always be shortcomings when attempting to quantify ecological metrics that are inherently dynamic in nature and have spatial and temporal dependencies that are challenging to account for. However, the Nearshore Calculator goes to great lengths to find and interpret published evidence and, in their absence, uses opinions of experts to establish a comprehensive evidence base for the tool... This expert review identified only limited shortcomings in the approach to populating the Calculator… Key recommendations are to use or develop more knowledge on the usage of the nearshore zone by juvenile and adult salmonids, especially adults, to better quantify the mortality impacts associated with habitat degradation."

Dr. Robert Lennox peer review (PDF, 24 pages)

Dr. Steven F. Railsback

Consulting Research Scientist, Lang, Railsback & Associates, Arcata, California and Adjunct Professor, Department of Mathematics California State Polytechnic University Humboldt, Arcata, California, USA

"I found the overall modeling approach appropriate and reasonable for its purpose. The Model combines key advantages of more conventional modeling approaches without their disadvantages… The Puget Sound model resembles habitat suitability models but uses mechanistic understanding and assumptions instead of empirical suitability relations. The basis in mechanisms, and the authors’ extensive reviews of supporting literature, provide a credible basis for model assumptions and weighting scores."

Dr. Steven F. Railsback peer review (PDF, 23 pages)

Materials Reviewed

The reviewers based their review on the document listed below (links embedded).

Main Documents

• Ehinger et al. 2023. The Puget Sound Nearshore Habitat Conservation Calculator – central document providing background and information supporting model structure and quantifications. We are working to publish this report through the NWFSC. If you would like a copy now, please email Stephanie.Ehinger@noaa.gov.
• Nearshore Calculator Excel Spreadsheet
• Annotated Nearshore Habitat Values Model – Excel Spreadsheet used to populate lookup tables in background tabs of the Nearshore Calculator.

Background Documents

• Nearshore Calculator User Guide
• Cereghino et al. 2023. Estimation of Typical High Intertidal Beach-Face Slope in Puget Sound. – Describes GIS layers developed to identify Highest Astronomical Tide (HAT) contour lines in Puget Sound as well as beach slopes.
• Ehinger et al. 2015. Historic document on previous model version.
• Lambert and Chamberlin. 2023. Beach Nourishment in Puget Sound: Status, Use, and Habitat Impacts
• Salish Sea Nearshore Programmatic (SSNP) Biological Opinion (PDF, 359 pages)

Take Away

The Nearshore Calculator got good marks as a science-informed analytical tool that produces valid and reliable results. The reviewers believed that it could be strengthened, and offered an array of improvements ranging from cosmetic to substantive. We categorized these recommendations and identified how and when we plan to incorporate them.

Next Steps

Recommendations included improvements to the (1) The Puget Sound Nearshore Habitat Conservation Calculator (Ehinger et al. 2023), the central document providing background and information supporting model structure and quantifications; (2) User Guide; and (3) to the Nearshore Calculator itself.

1. We are addressing the improvement suggestions for The Puget Sound Nearshore Habitat Conservation Calculator (Ehinger et al. 2023) in coordination with the Science Center and Restoration Center authors as part of publishing it as a NWFSC Tech Report.
2. NMFS provides User Guide & Nearshore Calculator updates annually, each February. Some User Guide updates are for clarity and ease of use, while others are in tandem with Nearshore Calculator improvements. We will begin addressing reviewer suggestions with our February 2024 update.

Nearshore Calculator Improvements

Reviewers added a variety of improvement suggestions including adding a sensitivity analysis, adding additional modules, and updating the beach nourishment assessment. For our February 2024 update, we are incorporating the results of the beach nourishment Science Sprint, Lambert and Chamberlin 2023. Through public meetings and information from Calculator Expert Office Hours, we will decide on which improvement suggestions to address next given time and resource constraints. We will be partnering with NOAA’s Northwest and Southwest Fisheries Science Centers in implementing improvement suggestions.