Data Management Plan
GUID: gov.noaa.nmfs.inport:48212 | Published / External
Data Management Plan
DMP Template v2.0.1 (2015-01-01)
Please provide the following information, and submit to the NOAA DM Plan Repository.Reference to Master DM Plan (if applicable)
As stated in Section IV, Requirement 1.3, DM Plans may be hierarchical. If this DM Plan inherits provisions from a higher-level DM Plan already submitted to the Repository, then this more-specific Plan only needs to provide information that differs from what was provided in the Master DM Plan.
1. General Description of Data to be Managed
This data set (Project Number OPR-I305-KRL-06) depicts depth values (mean 5 meter gridded) collected using LiDAR
(Light Detection & Ranging) from the shoreline of southwestern Puerto Rico to the shelf edge. The Tenix LADS Corporation
(TLI) acquired bathymetric LIDAR for NOAA from 4/07/2006 to 5/15/2006. Data was acquired with a LADS (Laser Airborne Depth Sounder)
Mk II Airborne System from altitudes between 1,200 and 2,200ft at ground speeds between 140 and 175 knots. The 900 Hertz Nd: YAG
(neodymium-doped yttrium aluminum garnet) laser (1064 nm) acquired 4x4 meter spot spacing and 200% seabed coverage.
This data set also contains LiDAR (Light Detection & Ranging) intensity values (mean 5 meter gridded) from the shoreline
of southwestern Puerto Rico to about 50 meters in depth. Reflectivity was calculated for each sounding as the ratio of
returned energy to transmitted energy, normalized for losses in a single wavelength (green/blue 532nm). The numerical
values for the relative reflectivity are scaled logarithmically to an 8-bit integer range 0 - 255
In total, 265 square nautical miles of LiDAR were collected between -50 m (topographic) and up to 70 m (depth), requiring a
total of 102 flight hours (134 hours, including flight time to and from San Juan airport). Environmental factors such
as wind strength and direction, cloud cover, and water clarity influenced the area of data acquisition on a daily basis.
The data was processed using the LADS Mk II Ground System and data visualization, quality control and final products were
created using CARIS HIPS and SIPS 6.1 and CARIS BASE Editor 2.1 The project was conducted to meet the IHO (International
Hydrographic Organization) Order 1 accuracy standards, dependent on the project area and depth. All users should individually
evaluate the suitability of this data according to their own needs and standards.
The data were received by the NOAA Office for Coastal Management from NCCOS. For data storage and Digital Coast provisioning
purposes the data were converted:
1. From MLLW depths to NAVD88 heights using Vdatum and then from NAVD88 heights to Ellipsoid heights using Geoid12a.
2. From UTM Zone 19, meters, NAD83 to geographic coordinates.
3. From no point classification to 11 (NOAA OCM bathymetry classification).
Under contract to the National Ocean Service, TLI collected airborne LiDAR bathymetry under Contract Number DG133C-03-CQ-0011,
Task Order T0008 and Project Number OPR-I305-KRL-06.
Original contact information:
Contact Org:
Department of Commerce (DOC), National Oceanic and Atmospheric Administration (NOAA), National Ocean Service (NOS),
Center for Coastal Monitoring and Assessment (CCMA), Biogeography Branch
Title: Project Manager - U.S. Caribbean
Phone: (301) 713-3028
Notes: Only a maximum of 4000 characters will be included.
Notes: Data collection is considered ongoing if a time frame of type "Continuous" exists.
Notes: All time frames from all extent groups are included.
Notes: All geographic areas from all extent groups are included.
(e.g., digital numeric data, imagery, photographs, video, audio, database, tabular data, etc.)
(e.g., satellite, airplane, unmanned aerial system, radar, weather station, moored buoy, research vessel, autonomous underwater vehicle, animal tagging, manual surveys, enforcement activities, numerical model, etc.)
2. Point of Contact for this Data Management Plan (author or maintainer)
Notes: The name of the Person of the most recent Support Role of type "Metadata Contact" is used. The support role must be in effect.
Notes: The name of the Organization of the most recent Support Role of type "Metadata Contact" is used. This field is required if applicable.
3. Responsible Party for Data Management
Program Managers, or their designee, shall be responsible for assuring the proper management of the data produced by their Program. Please indicate the responsible party below.
Notes: The name of the Person of the most recent Support Role of type "Data Steward" is used. The support role must be in effect.
4. Resources
Programs must identify resources within their own budget for managing the data they produce.
5. Data Lineage and Quality
NOAA has issued Information Quality Guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of information which it disseminates.
(describe or provide URL of description):
Process Steps:
- 2006-01-01 00:00:00 - Data Acquisition: For this project (OPR-I305-KRL-06), the Chief of Party was TLI's Darren Stephenson and Hydrographer was TLI's Mark Sinclair. Data was collected between 4/7/2006 & 5/15/2006 using the LADS Mk II Airborne System. The LADS Mk II Airborne System (AS) consists of a Dash 8-200 series aircraft, which has a transit speed of 250 knots at altitudes of up to 25,000ft and an endurance of up to eight hours. Survey operations are conducted from heights between 1,200 and 2,200ft at ground speeds between 140 and 175 knots. The aircraft was fitted with an Nd: YAG laser, which operates at 900 Hertz from a stabilized platform to provide a number of different spot spacings. The survey area was sounded at 4x4m laser spot spacing with main lines of sounding spaced at 80m, which provided the required 200% coverage. Green laser pulses are scanned beneath the aircraft in a rectilinear pattern. The pulses are reflected from the land, sea surface, within the water column and from the seabed. The height of the aircraft is determined by the infrared laser return, which is supplemented by the inertial height from the Attitude and Heading Reference System and GPS height. Real-time positioning is obtained by an Ashtech GG24 GPS receiver combined with Wide Area DGPS (Differential Global Positioning System) provided by the Fugro Omnistar to provide a differentially corrected position. Ashtech Z12 GPS receivers are also provided as part of the Airborne System and Ground Systems to log KGPS (Kinetic Global Positioning System) data on the aircraft and at a locally established GPS (Global Positioning System) base station. For more details on the airborne system, refer to the DAPR (Data Acquisition and Processing Report) at: https://noaa-nos-coastal-lidar-pds.s3.amazonaws.com/laz/geoid18/2518/supplemental/LiDAR_2006_PuertoRico_DAPR.pdf
- 2006-01-01 00:00:00 - Data Processing: The data was processed using the LADS Mk II Ground System. It consists of a portable Compaq Alpha ES40 Series 3 processor server with 1 GB EEC RAM, 764 GB disk space, digital linear tape (DLT) drives and magazines, digital audio tape (DAT) drive, CD ROM drive and is networked to up to 12 Compaq 1.5 GHz PCs and a HP 800ps Design Jet Plotter, printers and QC workstations. The GS supports survey planning, data processing, quality control and data export. The GS component also includes a KGPS base station, which provides independent post-processed position and height data. The LADS ground system includes a reflectivity algorithm in which reflectivity is calculated for each sounding as the ratio of returned energy to transmitted energy, normalized for losses. A comprehensive description of the GS is provided in the Data Acquisition and Processing Report (DAPR) available at: https://noaa-nos-coastal-lidar-pds.s3.amazonaws.com/laz/geoid18/2518/supplemental/LiDAR_2006_PuertoRico_DAPR.pdf Corrections to Soundings: The optics and electronics for laser transmission and reflected waveform collection for all soundings is done by equipment mounted on a stabilized platform within the aircraft. This platform is stabilized by an Attitude and Heading Reference System (AHRS) that minimizes the motion effect (roll and pitch) of the aircraft and all residuals from the local horizontal are logged by the Airborne System for correctional processing by the Ground System. Sounding depths and positions are determined in the Ground System from the raw waveform, aircraft height and platform attitude parameters as logged by the Airborne System. The Ground System automatically corrects soundings for aircraft height and heading, offsets between sensors, latency, mirror and platform angles, sea surface model errors, refraction of the laser beam at the sea surface, the effects of scattering of the beam in the water column and reduction for tide. Correct operation of the system is verified by static and dynamic position checks, benchmark lines and analysis of overlaps, redundancy from the 200% coverage of the seabed and cross line comparison results.
- 2006-01-01 00:00:00 - Reflectance Data: The reflectance XYZ data was imported into CARIS HIPS and SIPS as single beam data. The reflectance data was treated much the same as ordinary XYZ data (XY horizontal position and Z as reflectance value). The procedure used is as follows: - Modified XYZ file to add timestamp by using the CARIS XYZ File Manipulator utility - Created an import script using the CARIS Generic Data Parser - Imported modified XYZ file into CARIS HIPS and SIPS - Computed Total Propagated Error (TPE) using zero values - Applied tide corrections using a zero value file - Merged TPE and tide correction with data - Created field sheet to compute BASE surface of reflectance data The XYZ had to be modified to add timestamp to each data point. The XYZ file was divided into smaller files containing a maximum of 1,000,000 points per file. Each modified XYZ file was imported using the CARIS Generic Data Parser, creating an import script to recognize each field attribute. Two scripts were created to import timestamps 1000000 to 9999999 and the other 10000000 to 138000000. Once each XYZ file was imported into CARIS a generic zero tide value was applied to each modified XYZ file followed by a zero TPE value computation. The TPE and tide calculations were then merged with the reflectance data. A single field sheet was then created to incorporate six 'mean' BASE surfaces, gridded at a 5m resolution, which covered the entire survey area. Using CARIS BASE Editor, these six BASE surfaces were combined into one surface to produce a final 'mean' reflectance BASE surface. The necessity to divide the area into six initially was purely due to processing limitations. Also, the grid resolution does not change relative to depth, as the laser pulse footprint stays relatively constant regardless of depth and the laser spot spacing is consistent irrespective of aircraft altitude. The 5m grid provides the largest amount of detail that can be supported by the LiDAR data density.
- 2006-01-01 00:00:00 - Quality Control & Product Creation: CARIS HIPS and SIPS 6.1, Terramodel, Generic Mapping Tool, Visualization Tool Kit and Olex were used for data visualization, quality control and final product creation. Validation proceeds through the following steps: 1. Examining the Depth Profile for the correct processing of each expected Survey Run. 2. Examining the Position Confidence (C3) profile to verify that adequate position accuracy is maintained during the Survey Run. Note: Other profiles of supporting data such as EHE, number of satellites, and latency may also be examined as run profiles. 3. Examining the Coverage Confidence (C6) profile to verify that no coverage gaps exist in the Survey Run. Resolving anomalous soundings by examining data points in the Survey Run by checking: a. The Primary Depth Display b. The Waterfall Display c. The Waveform Display d. The Local Area Display Editing operations include selection of the alternate depth, assignment of NBA or deletion of the sounding as appropriate. Based on assessments made in the above steps the operator segments the line classifying each segment as: a. Accepted b. Anomalous (data not to be used) or c. Rejected (for refly) All operator interactions during the validation phase are logged so that complete traceability is maintained. Data Visualization - All validated and checked data is exported from the GS in a defined ASCII format for spatial presentation and checking. The position, depth, run and other relevant information are extracted from the line-based data for use in the generation of Triangulated Irregular Networks (TINs) and gridded data sets. Both of these are used to produce contour plots, sun-illuminated color banded images and coverage check plots. Anomalies found in these plots are reported back to the checkers for remedial action in the GS.
- 2006-01-01 00:00:00 - Data Gaps -The survey area was sounded at 4x4m laser spot spacing with main lines of sounding spaced at 80m, which provided the required 200% coverage. It should be noted that at 4x4m laser spot spacing, there is a gap of 1 to 1.5m between the illuminated areas of adjacent soundings at the sea surface. There is a possibility that small objects in shallow water along the coastline may fall between consecutive 4x4m soundings and not be detected. There are also some gaps in the data due to turbidity and very shallow water, as well as an intermittent laser problem on the last survey sortie. This has resulted in some along track and cross track anomalies and at the time this satisfied the requirement of the survey. Position Checks - Two independent positioning systems were used during the survey. Real-time positions were aided by WADGPS (Wide Area Differential Global Positioning System). A post-processed KGPS position was also determined relative to a local GPS base station that was established on the rooftop of the Courtyard Marriott Hotel in San Juan. The post-processed KGPS position solutions were applied to each sounding during post-processing and the height used in the datum filter. Horizontal Control - Data collection and processing were conducted on the Airborne and Ground Systems in World Geodetic System (WGS84) on Universal Transverse Mercator (Northern Hemisphere) projection UTM (N) in Zone 19, Central Meridian 69 W. All units are in meters. This data was post-processed and all soundings are relative to the North American Datum 1983 (NAD83). For more details, please see the Vertical and Horizontal Control Report. Water Clarity - The water clarity in the survey area was ideal for laser bathymetry as the water was very clear. Coverage was obtained for the majority of the survey area. The only area where coverage was not achieved was due to turbidity or very shallow water. Water depths to 50m were achieved at the extent of the predominant reef structure in SW Puerto Rico. The majority of the survey area is less than 20m deep. There are a number of areas throughout the survey area where no depths were achieved due to turbidity or very shallow water. The water clarity in some areas did vary on a daily basis, which required careful management. Additional survey lines were planned and flown to minimize the data gaps due to turbidity.
- 2007-12-01 00:00:00 - The final images were re-projected from Transverse Mercator to NAD 1983 UTM Zone 19N using ArcGIS. Holidays (i.e., missing data values) were filled using a nearest neighbor interpolation technique.
- 2013-07-01 00:00:00 - The NOAA Office for Coastal Management received the bathymetric lidar and reflectance data in two 5X5 m grids, in geotiff format. The data were in UTM Zone 19N, meters, NAD83 coordinates and were vertically referenced to MLLW. The vertical units of the data were meters. OCM performed the following processing for data storage and Digital Coast provisioning purposes: 1. The data (both bathymetry and reflectance) were converted from geotiff format to xyz text format. 2. The bathymetric data file was split up into 12 smaller files, using the linux command, split. 3. The 12 bathymetric files (xyz) and the reflectance file (xyr) were combined into 12 xyzr files using the NOAA OCM script, brundle. 4. The 12 xyzr files were processed through VDatum to convert from UTM coordinates to geographic coordinates and to convert from MLLW depths to NAVD88 heights. 5. The 12 xyzr files were converted to las format and the point classifications set to 11 (NOAA OCM bathymetry) using the lastools tool, txt2las 6. The data were converted from NAVD88 heights to ellipsoid heights using Geoid 12a and the geotiff keys were assigned. 7. Data were zipped to laz format
(describe or provide URL of description):
6. Data Documentation
The EDMC Data Documentation Procedural Directive requires that NOAA data be well documented, specifies the use of ISO 19115 and related standards for documentation of new data, and provides links to resources and tools for metadata creation and validation.
Missing/invalid information:
- 1.7. Data collection method(s)
- 3.1. Responsible Party for Data Management
- 4.1. Have resources for management of these data been identified?
- 4.2. Approximate percentage of the budget for these data devoted to data management
- 5.2. Quality control procedures employed
- 7.1. Do these data comply with the Data Access directive?
- 7.1.1. If data are not available or has limitations, has a Waiver been filed?
- 7.1.2. If there are limitations to data access, describe how data are protected
- 7.4. Approximate delay between data collection and dissemination
- 8.1. Actual or planned long-term data archive location
- 8.3. Approximate delay between data collection and submission to an archive facility
- 8.4. How will the data be protected from accidental or malicious modification or deletion prior to receipt by the archive?
(describe or provide URL of description):
7. Data Access
NAO 212-15 states that access to environmental data may only be restricted when distribution is explicitly limited by law, regulation, policy (such as those applicable to personally identifiable information or protected critical infrastructure information or proprietary trade information) or by security requirements. The EDMC Data Access Procedural Directive contains specific guidance, recommends the use of open-standard, interoperable, non-proprietary web services, provides information about resources and tools to enable data access, and includes a Waiver to be submitted to justify any approach other than full, unrestricted public access.
None, but please cite any use of this data.
Notes: The name of the Organization of the most recent Support Role of type "Distributor" is used. The support role must be in effect. This information is not required if an approved access waiver exists for this data.
Notes: This field is required if a Distributor has not been specified.
https://noaa-nos-coastal-lidar-pds.s3.amazonaws.com/laz/geoid18/2518/index.html
Notes: All URLs listed in the Distribution Info section will be included. This field is required if applicable.
This data can be obtained on-line at the following URL:
https://coast.noaa.gov/dataviewer/#/lidar/search/where:ID=2518
This data set is dynamically generated based on user-specified parameters.
;
Notes: This field is required if applicable.
8. Data Preservation and Protection
The NOAA Procedure for Scientific Records Appraisal and Archive Approval describes how to identify, appraise and decide what scientific records are to be preserved in a NOAA archive.
(Specify NCEI-MD, NCEI-CO, NCEI-NC, NCEI-MS, World Data Center (WDC) facility, Other, To Be Determined, Unable to Archive, or No Archiving Intended)
Notes: This field is required if archive location is World Data Center or Other.
Notes: This field is required if archive location is To Be Determined, Unable to Archive, or No Archiving Intended.
Notes: Physical Location Organization, City and State are required, or a Location Description is required.
Discuss data back-up, disaster recovery/contingency planning, and off-site data storage relevant to the data collection
9. Additional Line Office or Staff Office Questions
Line and Staff Offices may extend this template by inserting additional questions in this section.