Data Management Plan
GUID: gov.noaa.nmfs.inport:69463 | Published / External
Data Management Plan
DMP Template v2.0.1 (2015-01-01)
Please provide the following information, and submit to the NOAA DM Plan Repository.Reference to Master DM Plan (if applicable)
As stated in Section IV, Requirement 1.3, DM Plans may be hierarchical. If this DM Plan inherits provisions from a higher-level DM Plan already submitted to the Repository, then this more-specific Plan only needs to provide information that differs from what was provided in the Master DM Plan.
1. General Description of Data to be Managed
Product: These lidar data are processed Classified LAS 1.4 files, formatted to 2601 individual 1000 m x 1000 m tiles; used to create intensity images, 2D refraction extents, and Topobathy DEMs as necessary.
Geographic Extent: Collier, Monroe, and Miami-Dade counties, West Everglades, Florida, covering approximately 869 square miles.
Dataset Description: Florida West Everglades National Park 2018 Lidar project called for the Planning, Acquisition, processing and derivative products of lidar data to be collected at a nominal pulse spacing (NPS) of 0.35 meters. Project specifications are based on the U.S. Geological Survey National Geospatial Program Base Lidar Specification, Version 1.3. The data was developed based on a horizontal projection/datum of NAD83 (2011),Conus Albers, meters and vertical datum of NAVD88 (GEOID12B), meters. Lidar data were delivered as processed Classified LAS 1.4 files, formatted to 2601 individual 1000 m x 1000 m tiles, as tiled Intensity Imagery, and as tiled bare earth topobathy DEMs; all tiled to the same 1000 m x 1000 m schema.
Ground Conditions: Lidar was collected while no snow was on the ground and rivers were at or below normal levels. In order to post process the lidar data to meet task order specifications and meet ASPRS vertical accuracy guidelines, Leading Edge Geomatics established a total of 183 ground control points that were used to calibrate the lidar to known ground locations established throughout the Florida West Everglades National Park project area. Dewberry surveyed an additional 51 GCPs to test the calibrated swath data. Dewberry also surveyed 125 accuracy checkpoints to assess the vertical accuracy of the final data. Neither the Dewberry surveyed GCPs or checkpoints were used to calibrate or post process the data. Some surveyed points were placed in poor locations and had to be removed from accuracy testing. In order to meet ASPRS survey point requirements, 12 Dewberry surveyed GCPs were used in the final vertical accuracy testing. As the Dewberry surveyed GCPs were not used in any calibration processing and were only used to test calibrated data, all surveyed points used in final accuracy testing (Dewberry surveyed checkpoints and Dewberry surveyed GCPs) are an independent validation of the final calibrated, processed, and edited data. Additionally, one NVA point was actually located in vegetation so it was used to assess VVA despite its checkpoint ID name. A total of 113 surveyed points (55 NVA, 45 VVA, and 13 Bathymetric Bottom) were used in the final accuracy testing. This delivery is for the full project AOI and consists of 2601 lidar tiles.
This metadata record reflects the data that are available from the NOAA Digital Coast Data Access Viewer (DAV).
The NOAA Office for Coastal Management (OCM) downloaded 2601 laz point data files from this USGS site:
https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/FL_WestEvergladesNP_2018_B18/FL_WestEvergladesNP_topobathymetric_2018/LAZ/
The data were processed to the NOAA Digital Coast Data Access Viewer (DAV) to make the data available for custom downloads.
Notes: Only a maximum of 4000 characters will be included.
Notes: Data collection is considered ongoing if a time frame of type "Continuous" exists.
Notes: All time frames from all extent groups are included.
Notes: All geographic areas from all extent groups are included.
(e.g., digital numeric data, imagery, photographs, video, audio, database, tabular data, etc.)
(e.g., satellite, airplane, unmanned aerial system, radar, weather station, moored buoy, research vessel, autonomous underwater vehicle, animal tagging, manual surveys, enforcement activities, numerical model, etc.)
2. Point of Contact for this Data Management Plan (author or maintainer)
Notes: The name of the Person of the most recent Support Role of type "Metadata Contact" is used. The support role must be in effect.
Notes: The name of the Organization of the most recent Support Role of type "Metadata Contact" is used. This field is required if applicable.
3. Responsible Party for Data Management
Program Managers, or their designee, shall be responsible for assuring the proper management of the data produced by their Program. Please indicate the responsible party below.
Notes: The name of the Person of the most recent Support Role of type "Data Steward" is used. The support role must be in effect.
4. Resources
Programs must identify resources within their own budget for managing the data they produce.
5. Data Lineage and Quality
NOAA has issued Information Quality Guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of information which it disseminates.
(describe or provide URL of description):
Lineage Statement:
Data were collected and processed by Dewberry and Leading Edge Geomatics and were made available on the USGS ftp site. The data were downloaded from the USGS rockyweb site by the NOAA Office for Coastal Management (OCM) where the data were processed to make it available for custom download from the NOAA Digital Coast Data Access Viewer (DAV).
Process Steps:
- 2020-07-01 00:00:00 - The boresight for each lift was done individually as the solution may change slightly from lift to lift. The following steps describe the Raw Data Processing and Boresight process: 1) Technicians processed the raw data to LAS format flight lines using the final GPS/IMU solution. This LAS data set was used as source data for boresight. 2) Technicians first used commercial software to calculate initial boresight adjustment angles based on sample areas selected in the lift. These areas cover calibration flight lines collected in the lift, cross tie and production flight lines. These areas are well distributed in the lift coverage and cover multiple terrain types that are necessary for boresight angle calculation. The technician then analyzed the results and made any necessary additional adjustment until it is acceptable for the selected areas. 3) Once the boresight angle calculation was completed for the selected areas, the adjusted settings were applied to all of the flight lines of the lift and checked for consistency. The technicians utilized commercial and proprietary software packages to analyze how well flight line overlaps match for the entire lift and adjusted as necessary until the results met the project specifications. 4) Once all lifts were completed with individual boresight adjustment, the technicians checked and corrected the vertical misalignment of all flight lines and also the matching between data and ground truth. The relative accuracy was less than or equal to 6 cm RMSDz within individual swaths and less than or equal to 8 cm RMSDz or within swath overlap (between adjacent swaths). The sensors utilized on this project had both a NIR laser and a green laser so intraswath analyses, including intraswath polygon generation and review, were performed for each laser separately. The eastern add-on portion of the FL West Everglades NP project area was particularly difficult to align due to relatively few hard surfaces present which could reliably be used during the alignment process. Only one road is present in this add-on area. Due to the nature of relative swath alignment being based off the ground surface within each swath (which in reality is just the lowest plane of points), the process of using the lowest points can introduce some discrepancies due to these assumptions. In some areas one sensor may penetrate the vegetation to a greater extent, resulting in a lower last return surface. The alignment process bases the corrections from the statistical trends found in these offsets. This means that if there is a much greater coverage of vegetated areas than hard surfaces, those vegetated areas can have a much larger impact on the relative alignment of the data. In the case of this area the discrepancy resulted in a misalignment along a portion of the roadway while the vegetated areas nearby show no misalignment. Examining the park road in the east of the project shows that several of the swaths have some bias between the NIR and green swaths (approximately 7-11 cm), but there are 3-4 flightlines with larger offsets approaching 15 cm. Areas along the road exhibiting the most measureable offsets are identified in the provided shapefile, named W_Everglades_NP_Lidar_Interswath_Issues. Please see the project report for more details on interswath and intraswath testing and review. 5) The technicians ran a final vertical accuracy check of the boresighted flight lines against the surveyed check points to ensure the requirement of NVA = 19.6 cm 95% Confidence Level (Required Accuracy) is met. Point classification was performed according to USGS Lidar Base Specification 1.3 including the addition of bathy domain classes. Refraction extents (2D) were generated from the refracted lidar points. Topobathy DEMs were generated from the classified point cloud.
- 2020-09-01 00:00:00 - Automated grounding was performed using Terrascan software. This routine classifies any obvious low outliers in the dataset to class 7 and high outliers in the dataset to class 18. Points along flight line edges that are geometrically unusable are identified by their scan angle and classified to a separate class so that they will not be used in the initial ground algorithm. These point with higher scan angles will be set to withheld later in the lidar processing. After points that could negatively affect the ground are removed from class 1, the ground layer is extracted from this remaining point cloud. The ground extraction process encompassed in this routine takes place by building an iterative surface model. The final refraction extents are then used to classify ground points within the refraction extents as bathymetric bottom. The refraction extents are also used as part of the classification routines to ensure water surface and water column points are classified correctly. Each tile was then imported into Terrascan and a surface model was created to examine the ground (class 2) and bathy bottom (class 40) classification. Dewberry analysts employ 3D visualization techniques to view the point cloud at multiple angles and in profile to ensure that non-ground points are removed from the ground classification and that class 40 accurately represents submerged topography. Dewberry analysts visually reviewed the surface models and corrected errors in the ground classification such as vegetation, buildings, and bridges that were present following the initial processing conducted by Dewberry. Bridge decks are manually classified to class 17. The withheld bit is set on the points with higher scan angles previously identified in Terrascan before the ground classification routine was performed. After manual classification, the LAS tiles were peer reviewed and then underwent a final QA/QC. After the final QA/QC and corrections, all headers, appropriate point data records, and variable length records, including spatial reference information, are updated in proprietary software and then verified using proprietary Dewberry tools.
- 2020-08-01 00:00:00 - Due to two lasers being utilized per each sensor and the high amount of overlap, the average calculated (A)NPD is much hihger than the project requirements of 0.35 meter NPS and 8 points per square meter (ppsm). A portion of this project was funded after the original Task Order, as an add-on. The average calculated (A)NPD of the original AOI is 23 ppsm (0.21 m NPS) and the average (A)NPD of the add-on portion of this AOI is 14 ppsm (0.27 m NPS). The average density was tested on the LAS data using geometrically reliable (withheld and noise points excluded) first-return points. (A)NPD was tested using rasters which calculate the average number of points within each cell.
- 2023-03-22 00:00:00 - The NOAA Office for Coastal Management (OCM) downloaded 2601 laz point data files from this USGS site: https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/FL_WestEvergladesNP_2018_B18/FL_WestEvergladesNP_topobathymetric_2018/LAZ/ The data were in Albers Equal Area (NAD83 2011), meters coordinates and NAVD88 (Geoid12B) elevations in meters. The data were classified as: 1 - Unclassified, 2 - Ground, 7 - Low Noise, 17 - Bridge Decks, 18 - High Noise, 40 - Bathymetric Bottom, 41 - Water Surface, 45 - No bathymetric bottom found (water column). OCM processed all classifications of points to the Digital Coast Data Access Viewer (DAV). Classes available on the DAV are: 1, 2, 7, 17, 18, 40, 41, 45. OCM performed the following processing on the data for Digital Coast storage and provisioning purposes: 1. Internal OCM scripts were run to check the number of points by classification and by flight ID and the gps, elevation, and intensity ranges. 2. Internal OCM scripts were run on the laz files to: a. Convert from orthometric (NAVD88) elevations to NAD83 (2011) ellipsoid elevations using the Geoid12B model b. Convert the laz files from Albers Equal Area (NAD83 2011), meters coordinates to geographic coordinates c. Assign the geokeys, sort the data by gps time and zip the data to database.
(describe or provide URL of description):
6. Data Documentation
The EDMC Data Documentation Procedural Directive requires that NOAA data be well documented, specifies the use of ISO 19115 and related standards for documentation of new data, and provides links to resources and tools for metadata creation and validation.
Missing/invalid information:
- 1.7. Data collection method(s)
- 3.1. Responsible Party for Data Management
- 5.2. Quality control procedures employed
- 7.1.1. If data are not available or has limitations, has a Waiver been filed?
- 7.4. Approximate delay between data collection and dissemination
- 8.3. Approximate delay between data collection and submission to an archive facility
(describe or provide URL of description):
7. Data Access
NAO 212-15 states that access to environmental data may only be restricted when distribution is explicitly limited by law, regulation, policy (such as those applicable to personally identifiable information or protected critical infrastructure information or proprietary trade information) or by security requirements. The EDMC Data Access Procedural Directive contains specific guidance, recommends the use of open-standard, interoperable, non-proprietary web services, provides information about resources and tools to enable data access, and includes a Waiver to be submitted to justify any approach other than full, unrestricted public access.
None
Notes: The name of the Organization of the most recent Support Role of type "Distributor" is used. The support role must be in effect. This information is not required if an approved access waiver exists for this data.
Notes: This field is required if a Distributor has not been specified.
https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/FL_WestEvergladesNP_2018_B18/FL_WestEvergladesNP_topobathymetric_2018/LAZ/
Notes: All URLs listed in the Distribution Info section will be included. This field is required if applicable.
Data is available online for bulk and custom downloads.
Notes: This field is required if applicable.
8. Data Preservation and Protection
The NOAA Procedure for Scientific Records Appraisal and Archive Approval describes how to identify, appraise and decide what scientific records are to be preserved in a NOAA archive.
(Specify NCEI-MD, NCEI-CO, NCEI-NC, NCEI-MS, World Data Center (WDC) facility, Other, To Be Determined, Unable to Archive, or No Archiving Intended)
Notes: This field is required if archive location is World Data Center or Other.
Notes: This field is required if archive location is To Be Determined, Unable to Archive, or No Archiving Intended.
Notes: Physical Location Organization, City and State are required, or a Location Description is required.
Discuss data back-up, disaster recovery/contingency planning, and off-site data storage relevant to the data collection
Data is backed up to tape and to cloud storage.
9. Additional Line Office or Staff Office Questions
Line and Staff Offices may extend this template by inserting additional questions in this section.