Data Management Plan
GUID: gov.noaa.nmfs.inport:66821 | Published / External
Data Management Plan
DMP Template v2.0.1 (2015-01-01)Please provide the following information, and submit to the NOAA DM Plan Repository.
Reference to Master DM Plan (if applicable)
As stated in Section IV, Requirement 1.3, DM Plans may be hierarchical. If this DM Plan inherits provisions from a higher-level DM Plan already submitted to the Repository, then this more-specific Plan only needs to provide information that differs from what was provided in the Master DM Plan.
1. General Description of Data to be Managed
Topo-bathy lidar acquisition and processing in the Mariana Islands covering Aguijan, Rota, Saipan, and Tinian. This data is the integrated topo-bathy DEM derived from classes 2, 40, and 43, delivered in 500m x 500m tiles as 1m GeoTIFFs with no data values set to -999999 and bridges removed. Not hydro-flattened.
In addition to these topobathy Digital Elevation Model (DEM) data, the lidar point data that these DEM data were created from and the topo hydro-flattened version of the DEM are also available. These data are available for download at the links provided in the URL section of this metadata record.
Notes: Only a maximum of 4000 characters will be included.
Notes: Data collection is considered ongoing if a time frame of type "Continuous" exists.
Notes: All time frames from all extent groups are included.
Notes: All geographic areas from all extent groups are included.
(e.g., digital numeric data, imagery, photographs, video, audio, database, tabular data, etc.)
(e.g., satellite, airplane, unmanned aerial system, radar, weather station, moored buoy, research vessel, autonomous underwater vehicle, animal tagging, manual surveys, enforcement activities, numerical model, etc.)
2. Point of Contact for this Data Management Plan (author or maintainer)
Notes: The name of the Person of the most recent Support Role of type "Metadata Contact" is used. The support role must be in effect.
Notes: The name of the Organization of the most recent Support Role of type "Metadata Contact" is used. This field is required if applicable.
3. Responsible Party for Data Management
Program Managers, or their designee, shall be responsible for assuring the proper management of the data produced by their Program. Please indicate the responsible party below.
Notes: The name of the Person of the most recent Support Role of type "Data Steward" is used. The support role must be in effect.
Programs must identify resources within their own budget for managing the data they produce.
5. Data Lineage and Quality
NOAA has issued Information Quality Guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of information which it disseminates.
(describe or provide URL of description):
This data was collected by Woolpert, Inc. for the USGS and NOAA Office for Coastal Management (OCM).
- All lidar data were acquired using a HE4X sensor (Figure 4). The HE4X is a latest generation topographic and bathymetric lidar sensor. The system provides denser data than previous traditional bathymetric lidar systems. It is unique in its ability to acquire bathymetric lidar, topographic lidar and 4-band digital camera imagery simultaneously. The HE4X provided up to 500 kHz topographic data and an effective 140 kHz shallow bathymetric data and a 40 kHz deep channel. While not a required deliverable for this survey, 4-band 80 MP digital camera imagery was also collected simultaneously with the sensor's RCD-30 camera and utilized during data editing in some cases. The bathymetric and topographic lasers are independent and do not share an optical chain or receivers, so they are optimized for their specific function. As with any bathymetric lidar, maximum depth penetration is a function of water clarity and seabed reflectivity. The HE4X is designed to penetrate to 3 times the secchi depth. This is also represented as Dmax = 4/K, where K is the diffuse attenuation coefficient, and assuming K is between 0.1 and 0.3, a normal sea state and 15% seabed reflectance. Both the topographic and bathymetric sub-systems use a palmer scanner to produce an elliptical scan pattern of laser points with a degree of incidence ranging from +/-14 degrees (front and back) to +/-20 degrees (sides), providing a 40 degree field of view. This has the benefit of providing multiple look angles on a single pass and helps to eliminate shadowing effects. This can be of particular use in urban areas, where all sides of a building are illuminated, or for bathymetric features such as the sides of narrow water channels, or features on the seafloor such as smaller objects and wrecks. It also assists with penetration in the surf zone where the back scan passes the same ground location a couple of seconds after the front scan, allowing the areas of whitewater to shift.
- Position and orientation data were acquired in the aircraft using a NovAtel SPAN with LCI-100C IMU. All data were post-processed using NovAtel Inertial Explorer software to provide a tightly coupled position and orientation solution. A single base station was used to control trajectory processing providing final trajectories for Saipan and Tinian on NAD83 (MA11), Epoch 2010, located in the Saipan airport. This base station was replaced for each of the three separate collects of the project (Table 11). SPN1, SPN2 and SPN3 were occupied with a Trimble GNSS receiver by Woolpert. Due to the distance of Rota, Aguijan, Farallon de Medinilla, and Pagan from the single base station on Saipan and their remoteness a precise point positioning (PPP) solution was used for them on ITRF2014. To establish a reliable coordinate for SPN1 data were uploaded to the National Geodetic Service (NGS) Online Positioning User Service (OPUS), and for SPN2 and SPN3 Trimble CenterPoint RTX Post-Processing service was used. The average OPUS or RTX coordinate from multiple days of observations was used to process the final trajectories.
- Initial data coverage analysis and quality checks to ensure there were no potential system issues were carried out in the field prior to demobilization of the sensor. Final processing was conducted in Woolpert's offices. In general data were initially processed in Leica's Lidar Survey Studio (LSS) using final processed trajectory information. LAS files from LSS were then imported to a Terrascan project where spatial algorithms were used to remove noise and classify bare earth/ground. Manual review was conducted in both Terrascan and LP360 prior to product creation.
- Lidar processing was conducted using the Leica Lidar Survey Studio (LSS) software. Calibration information, along with processed trajectory information were combined with the raw laser data to create an accurately georeferenced lidar point cloud for the entire survey in LAS v1.4 format. All points from the topographic and bathymetric laser include 16-bit intensity values. During this LSS processing stage, an automatic land/water discrimination is made for the bathymetric waveforms. This allows the bathymetric (green) pulses over water to be automatically refracted for the pulse hitting the water surface and travelling through the water column, producing the correct depth. Another advantage of the automatic land/water discrimination is that it permits calculation of an accurate water surface over smaller areas, allowing simple bathymetric processing of smaller, narrower streams and drainage channels. Sloping water surfaces are also handled correctly. Prior to processing, the hydrographer can adjust waveform sensitivity settings dependent on the environment encountered and enter a value for the refraction index to be used for bathymetry. The index of refraction is an indication of the water type. Values used for sensitivity settings and the index of refraction are included in the LSS processing settings files. A value of 1.34206 was used for the index of refraction, indicating saltwater. In the field, default waveform sensitivity settings were used for processing. In order to determine the optimal waveform sensitivity settings for final processing, sample areas were selected and processed with multiple different settings, to iteratively converge on the best possible settings. This is done by reviewing the processed point cloud and waveforms within sample areas. A sample waveform is provided in Figure 6, while a sample LSS editing screen is provided in Figure 7. Settings affect which waveform peaks are classified as valid seabed, and which peaks are classified as noise. Optimal settings strike a balance between the amount of valid data that is classified as seabed bottom, and the amount of noise that is incorrectly classified due to peaks in the waveforms. Ideally all valid data is selected, while only a small amount of noise remains to be edited out. Once optimal threshold settings were chosen, these were used for the entire project. It is important to note that all digitized waveform peaks are available to be reviewed by the hydrographer; both valid seabed bottom and peaks classed as noise. This allows the hydrographer to review data during TerraScan and LP360 editing for valid data such as objects that may have been misclassified as noise. LSS processing produced LAS files in 1.4 format. Additional QC steps were performed prior to import to TerraScan. Firstly, the derived water surface was reviewed to ensure a water surface was correctly calculated for all bathymetry channels. No significant issues were apparent. Spot checks were also made on the data to ensure the front and back of the scans remained in alignment and no calibration or system issues were apparent prior to further data editing in TerraScan. LSS stores data in multiple LAS files for a single flight line. Each file corresponds to a single .dat file from the raw airborne data. Woolpert merged these multiple files into a single file per flight line and moved data into a standard class definition in preparation for data editing using Woolpert's proprietary scripts within SAFE's FME software. Data produced by LSS for flights over Saipan and Tinian were processed on the NAD83 (MA11) Epoch 2010 datum in UTM 55N Zone with units in meters, and elevations on the ellipsoid also in meters. Data produced for Aguijan and Rota were processed on the ITRF2014 datum in UTM 55N Zone with units in meters, with elevations on the ellipsoid also in meters.
- After data were processed in LSS and the data integrity reviewed, Aguijan, and Rota were transformed from the ITRF2014 ellipsoid to the NAD83 (MA11) Epoch 2010 ellipsoid using VDatum. With the entire project now on the correct ellipsoid, data were organized into tiles within a TerraScan project. The tile layout is the same as that provided with the project deliverables. Data classification and spatial algorithms were applied in Terrasolid's TerraScan software. Customized spatial algorithms, such as isolated points and low point filters, were run to remove gross fliers in the topographic and bathymetric data. A grounding algorithm was also run on the topographic data to distinguish between points representing the bare earth, and other valid topo lidar points representing features such as vegetation, buildings, and so forth. Algorithms were run on the entire dataset. Data were reviewed manually to reclassify any valid bathy points incorrectly identified by the automated routines in LSS as invalid, and vice versa. In addition, any topo points over the water were reclassified to correct the ground representation. Manual editing was conducted both in TerraScan and LP360. Steps for manual editing included: - Re-class any topo unclassified laser data and bathy seabed data from the water surface to a water surface class - Review bathymetry in cross section. - Re-class suitable data to Seabed (Class 40). - Re-class any noise in the bathy ground class to bathy noise (Class 45). - Review topo ground points in areas of gaps or spikes. - Add points to ground (Class 2) from the topo laser if points are available to fill gaps in the ground model. - Re-class any noise in the ground class to Topo Unclassified (Class 1) if valid vegetation or other feature, or Noise if the point is not valid (Low Noise (Class 7) or High Noise (Class 18)). - Review topo ground points for bridges and re-class to Bridge Deck (Class 17). - Review bathymetry using imagery and nautical charts and re-class obvious man-made objects to Submerged Object (Class 43). Once editing was completed in TerraScan the islands of Saipan, Tinian, Aguijan, and Rota were vertically transformed to the NMVD03 datum using GEOID12B.
- Digital Elevation Models (DEMs) were then created using TerraModel at 1m resolution using Topo Ground, Seabed, and Submerged Object classes. (Class 2, 40, and 43).
- 2022-03-11 00:00:00 - The NOAA Office for Coastal Management (OCM) received the raster DEM data in GeoTiff format from Woolpert, Inc. The data for the islands of Aguijan, Rota, Saipan, and Tinian were in UTM Zone 55 NAD83(MA11), meters coordinates and NMVD03 (Geoid 12B) elevations in meters. The bare earth raster files were at a 1 meter grid spacing. OCM performed the following processing on the data for Digital Coast storage and provisioning purposes: 1. Used internal an script to assign the EPSG codes (Horizontal EPSG: 8693 and Vertical EPSG: 6640) to the GeoTiff formatted files. 2. Copied the files to https.
(describe or provide URL of description):
6. Data Documentation
The EDMC Data Documentation Procedural Directive requires that NOAA data be well documented, specifies the use of ISO 19115 and related standards for documentation of new data, and provides links to resources and tools for metadata creation and validation.
- 1.7. Data collection method(s)
- 3.1. Responsible Party for Data Management
- 4.1. Have resources for management of these data been identified?
- 4.2. Approximate percentage of the budget for these data devoted to data management
- 5.2. Quality control procedures employed
- 7.1. Do these data comply with the Data Access directive?
- 7.1.1. If data are not available or has limitations, has a Waiver been filed?
- 7.1.2. If there are limitations to data access, describe how data are protected
- 7.4. Approximate delay between data collection and dissemination
- 8.1. Actual or planned long-term data archive location
- 8.3. Approximate delay between data collection and submission to an archive facility
- 8.4. How will the data be protected from accidental or malicious modification or deletion prior to receipt by the archive?
(describe or provide URL of description):
7. Data Access
NAO 212-15 states that access to environmental data may only be restricted when distribution is explicitly limited by law, regulation, policy (such as those applicable to personally identifiable information or protected critical infrastructure information or proprietary trade information) or by security requirements. The EDMC Data Access Procedural Directive contains specific guidance, recommends the use of open-standard, interoperable, non-proprietary web services, provides information about resources and tools to enable data access, and includes a Waiver to be submitted to justify any approach other than full, unrestricted public access.
Notes: The name of the Organization of the most recent Support Role of type "Distributor" is used. The support role must be in effect. This information is not required if an approved access waiver exists for this data.
Notes: This field is required if a Distributor has not been specified.
Notes: All URLs listed in the Distribution Info section will be included. This field is required if applicable.
Data is available online for bulk and custom downloads.
Notes: This field is required if applicable.
8. Data Preservation and Protection
The NOAA Procedure for Scientific Records Appraisal and Archive Approval describes how to identify, appraise and decide what scientific records are to be preserved in a NOAA archive.
(Specify NCEI-MD, NCEI-CO, NCEI-NC, NCEI-MS, World Data Center (WDC) facility, Other, To Be Determined, Unable to Archive, or No Archiving Intended)
Notes: This field is required if archive location is World Data Center or Other.
Notes: This field is required if archive location is To Be Determined, Unable to Archive, or No Archiving Intended.
Notes: Physical Location Organization, City and State are required, or a Location Description is required.
Discuss data back-up, disaster recovery/contingency planning, and off-site data storage relevant to the data collection
9. Additional Line Office or Staff Office Questions
Line and Staff Offices may extend this template by inserting additional questions in this section.