Data Management Plan
GUID: gov.noaa.nmfs.inport:50159 | Published / External
Data Management Plan
DMP Template v2.0.1 (2015-01-01)
Please provide the following information, and submit to the NOAA DM Plan Repository.Reference to Master DM Plan (if applicable)
As stated in Section IV, Requirement 1.3, DM Plans may be hierarchical. If this DM Plan inherits provisions from a higher-level DM Plan already submitted to the Repository, then this more-specific Plan only needs to provide information that differs from what was provided in the Master DM Plan.
1. General Description of Data to be Managed
Watershed Sciences, Inc. (WS) co-acquired Light Detection and Ranging (LiDAR) data and Truecolor
Orthophotographs of the Snohomish River Estuary, WA on July 20 & 21, 2009. The
original requested survey area (26,150 acres) was expanded, at the client's request, to
include more of the valley lowland areas in the SW and SE edge of the original AOI as well as
additional creeks on the northern edge of the survey (Figure 1). The total area of delivered
LiDAR and True-color Orthophotographs, including the expansion and 100 m buffer, is 32,140 acres.
Notes: Only a maximum of 4000 characters will be included.
Notes: Data collection is considered ongoing if a time frame of type "Continuous" exists.
Notes: All time frames from all extent groups are included.
Notes: All geographic areas from all extent groups are included.
(e.g., digital numeric data, imagery, photographs, video, audio, database, tabular data, etc.)
(e.g., satellite, airplane, unmanned aerial system, radar, weather station, moored buoy, research vessel, autonomous underwater vehicle, animal tagging, manual surveys, enforcement activities, numerical model, etc.)
2. Point of Contact for this Data Management Plan (author or maintainer)
Notes: The name of the Person of the most recent Support Role of type "Metadata Contact" is used. The support role must be in effect.
Notes: The name of the Organization of the most recent Support Role of type "Metadata Contact" is used. This field is required if applicable.
3. Responsible Party for Data Management
Program Managers, or their designee, shall be responsible for assuring the proper management of the data produced by their Program. Please indicate the responsible party below.
Notes: The name of the Person of the most recent Support Role of type "Data Steward" is used. The support role must be in effect.
4. Resources
Programs must identify resources within their own budget for managing the data they produce.
5. Data Lineage and Quality
NOAA has issued Information Quality Guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of information which it disseminates.
(describe or provide URL of description):
Process Steps:
- Point Generation. The points are generated as Terrascan binary Format using Terrapoint's proprietary Laser Postprocessor Software. This software combines the Raw Laser file and GPS/IMU information to generate a point cloud for each individual flight. All the point cloud files encompassing the project area were then divided into quarter quad tiles. The referencing system of these tiles is based upon the project boundary minimum and maximums. This process is carried out in Terrascan. The bald earth is subsequently extracted from the raw LiDAR points using Terrascan in a Microstation environment. The automated vegetation removal process takes place by building an iterative surface model. This surface model is generated using three main parameters: Building size, Iteration angle and Iteration distance. The initial model is based upon low points selected by a roaming window and are assumed to be ground points. The size of this roaming window is determined by the building size parameter. These low points are triangulated and the remaining points are evaluated and subsequently added to the model if they meet the Iteration angle and distance constraints (fig. 1). This process is repeated until no additional points are added within an iteration. There is also a maximum terrain angle constraint that determines the maximum terrain angle allowed within the model. Multiple process dates, report compiled 20050331.
- Applications and Work Flow Overview 1. Resolved kinematic corrections for aircraft position data using kinematic aircraft GPS and static ground GPS data. Software: Waypoint GPS v.8.10, Trimble Geomatics Office v.1.62 2. Developed a smoothed best estimate of trajectory (SBET) file that blends post-processed aircraft position with attitude data Sensor head position and attitude were calculated throughout the survey. The SBET data were used extensively for laser point processing. Software: IPAS v.1.4 3. Calculated laser point position by associating SBET position to each laser point return time, scan angle, intensity, etc. Created raw laser point cloud data for the entire survey in *.las (ASPRS v1.1) format. Software: ALS Post Processing Software v.2.69 4. Imported raw laser points into manageable blocks (less than 500 MB) to perform manual relative accuracy calibration and filter for pits/birds. Ground points were then classified for individual flight lines (to be used for relative accuracy testing and calibration). Software: TerraScan v.9.001 5. Using ground classified points per each flight line, the relative accuracy was tested. Automated line-to-line calibrations were then performed for system attitude parameters (pitch, roll, heading), mirror flex (scale) and GPS/IMU drift. Calibrations were performed on ground classified points from paired flight lines. Every flight line was used for relative accuracy calibration. Software: TerraMatch v.9.001 6. Position and attitude data were imported. Resulting data were classified as ground and nonground points. Statistical absolute accuracy was assessed via direct comparisons of ground classified points to ground RTK survey data. Data were then converted to orthometric elevations (NAVD88) by applying a Geoid03 correction. Ground models were created as a triangulated surface and exported as ArcInfo ASCII grids at a 1-meter pixel resolution. Software: TerraScan v.9.001, ArcMap v9.3, TerraModeler v.9.001 Laser point coordinates were computed using the IPAS and ALS Post Processor software suites based on independent data from the LiDAR system (pulse time, scan angle), and aircraft trajectory data (SBET). Laser point returns (first through fourth) were assigned an associated (x, y, z) coordinate along with unique intensity values (0-255). The data were output into large LAS v. 1.2 files; each point maintains the corresponding scan angle, return number (echo), intensity, and x, y, z (easting, northing, and elevation) information. These initial laser point files were too large for subsequent processing. To facilitate laser point processing, bins (polygons) were created to divide the dataset into manageable sizes (< 500 MB). Flightlines and LiDAR data were then reviewed to ensure complete coverage of the survey area and positional accuracy of the laser points. Laser point data were imported into processing bins in TerraScan, and manual calibration was performed to assess the system offsets for pitch, roll, heading and scale (mirror flex). Using a geometric relationship developed by Watershed Sciences, each of these offsets was resolved and corrected if necessary.
- LiDAR points were then filtered for noise, pits (artificial low points) and birds (true birds as well as erroneously high points) by screening for absolute elevation limits, isolated points and height above ground. Each bin was then manually inspected for remaining pits and birds and spurious points were removed. In a bin containing approximately 7.5-9.0 million points, an average of 50-100 points are typically found to be artificially low or high. Common sources of non-terrestrial returns are clouds, birds, vapor, haze, decks, brush piles, etc. LiDAR Data Acquisition and Processing: Snohomish River Estuary, WA Prepared by Watershed Sciences, Inc. Internal calibration was refined using TerraMatch. Points from overlapping lines were tested for internal consistency and final adjustments were made for system misalignments (i.e., pitch, roll, heading offsets and scale). Automated sensor attitude and scale corrections yielded 3-5 cm improvements in the relative accuracy. Once system misalignments were corrected, vertical GPS drift was then resolved and removed per flight line, yielding a slight improvement (<1 cm) in relative accuracy. The TerraScan software suite is designed specifically for classifying near-ground points (Soininen, 2004). The processing sequence began by 'removing' all points that were not 'near' the earth based on geometric constraints used to evaluate multi-return points. The resulting bare earth (ground) model was visually inspected and additional ground point modeling was performed in site-specific areas to improve ground detail. This manual editing of grounds often occurs in areas with known ground modeling deficiencies, such as: bedrock outcrops, cliffs, deeply incised stream banks, and dense vegetation. In some cases, automated ground point classification erroneously included known vegetation (i.e., understory, low/dense shrubs, etc.). These points were manually reclassified as non-grounds. Ground surface rasters were developed from triangulated irregular networks (TINs) of ground points.
- 2013-11-14 00:00:00 - The NOAA Office for Coastal Management (OCM) downloaded topographic files in text format from PSLC's website. The files contained lidar easting, northing, elevation, intensity, return number, class, scan angle and GPS time measurements. The data were received in Washington State Plane North Zone 4601, NAD83 coordinates and were vertically referenced to NAVD88 using the Geoid03 model. The vertical units of the data were feet. OCM performed the following processing for data storage and Digital Coast provisioning purposes: 1. The All-Return ASCII txt files were parsed to convert GPS Week Time to Adjusted Standard GPS Time. 2. The All-Return ASCII files were converted from txt format to las format using LASTools' txt2las tool and reclassified to fit the OCM class list, N=1 (unclassified), G=2 (ground). 3. The las files were converted from orthometric (NAVD88) heights to ellipsoidal heights using Geoid03. 4. The las files' vertical units were converted from feet to meters, removing bad elevations. 5. The las files were converted from a Projected Coordinate System (WA SP North) to a Geographic Coordinate system (NAD83) 6. The las files' horizontal units were converted from feet to decimal degrees and converted to laz format. 7. The laz tiles containing only water areas were removed and remaining tiles were clipped to remove excess noise.
(describe or provide URL of description):
6. Data Documentation
The EDMC Data Documentation Procedural Directive requires that NOAA data be well documented, specifies the use of ISO 19115 and related standards for documentation of new data, and provides links to resources and tools for metadata creation and validation.
Missing/invalid information:
- 1.7. Data collection method(s)
- 3.1. Responsible Party for Data Management
- 4.1. Have resources for management of these data been identified?
- 4.2. Approximate percentage of the budget for these data devoted to data management
- 5.2. Quality control procedures employed
- 7.1. Do these data comply with the Data Access directive?
- 7.1.1. If data are not available or has limitations, has a Waiver been filed?
- 7.1.2. If there are limitations to data access, describe how data are protected
- 7.4. Approximate delay between data collection and dissemination
- 8.1. Actual or planned long-term data archive location
- 8.3. Approximate delay between data collection and submission to an archive facility
- 8.4. How will the data be protected from accidental or malicious modification or deletion prior to receipt by the archive?
(describe or provide URL of description):
7. Data Access
NAO 212-15 states that access to environmental data may only be restricted when distribution is explicitly limited by law, regulation, policy (such as those applicable to personally identifiable information or protected critical infrastructure information or proprietary trade information) or by security requirements. The EDMC Data Access Procedural Directive contains specific guidance, recommends the use of open-standard, interoperable, non-proprietary web services, provides information about resources and tools to enable data access, and includes a Waiver to be submitted to justify any approach other than full, unrestricted public access.
None
Notes: The name of the Organization of the most recent Support Role of type "Distributor" is used. The support role must be in effect. This information is not required if an approved access waiver exists for this data.
Notes: This field is required if a Distributor has not been specified.
https://noaa-nos-coastal-lidar-pds.s3.amazonaws.com/laz/geoid18/2590/index.html
Notes: All URLs listed in the Distribution Info section will be included. This field is required if applicable.
This data can be obtained on-line at the following URL:
https://coast.noaa.gov/dataviewer/#/lidar/search/where:ID=2590
;
Notes: This field is required if applicable.
8. Data Preservation and Protection
The NOAA Procedure for Scientific Records Appraisal and Archive Approval describes how to identify, appraise and decide what scientific records are to be preserved in a NOAA archive.
(Specify NCEI-MD, NCEI-CO, NCEI-NC, NCEI-MS, World Data Center (WDC) facility, Other, To Be Determined, Unable to Archive, or No Archiving Intended)
Notes: This field is required if archive location is World Data Center or Other.
Notes: This field is required if archive location is To Be Determined, Unable to Archive, or No Archiving Intended.
Notes: Physical Location Organization, City and State are required, or a Location Description is required.
Discuss data back-up, disaster recovery/contingency planning, and off-site data storage relevant to the data collection
9. Additional Line Office or Staff Office Questions
Line and Staff Offices may extend this template by inserting additional questions in this section.