Data Management Plan
GUID: gov.noaa.nmfs.inport:49773 | Published / External
Data Management Plan
DMP Template v2.0.1 (2015-01-01)
Please provide the following information, and submit to the NOAA DM Plan Repository.Reference to Master DM Plan (if applicable)
As stated in Section IV, Requirement 1.3, DM Plans may be hierarchical. If this DM Plan inherits provisions from a higher-level DM Plan already submitted to the Repository, then this more-specific Plan only needs to provide information that differs from what was provided in the Master DM Plan.
1. General Description of Data to be Managed
The Concord AOI consists of one area. Ground Control is collected throughout the AOI for use in the processing of LiDAR data to ensure data accurately represents the ground surface. QA/QC checkpoints,
(FVA and CVA - see Ground Control process step for further information) also collected throughout the AOI, are used for independent quality checks of the processed LiDAR data.
LiDAR acquisition products include Pre- and Post- flight reports which contain information on the flightlines, equipment parameters, and other pertinant acquisition details. The LiDAR product
is considered to be point cloud data and consists of 1500mx1500m tiles of LAS points which are partially classified such that the bare earth points can be calibrated to the ground surface and
tested via the independent QC to ensure the ground surface is accurately represented. The Bare Earth deliverables consists of tiles of fully classified LAS points. A suite of products derived
from the LiDAR data including DEMs and contour data has been created to support H&H analysis within the Concord watershed. A full narrative accompanies this deliverable, as well as the independent QC report.
Notes: Only a maximum of 4000 characters will be included.
Notes: Data collection is considered ongoing if a time frame of type "Continuous" exists.
Notes: All time frames from all extent groups are included.
Notes: All geographic areas from all extent groups are included.
(e.g., digital numeric data, imagery, photographs, video, audio, database, tabular data, etc.)
(e.g., satellite, airplane, unmanned aerial system, radar, weather station, moored buoy, research vessel, autonomous underwater vehicle, animal tagging, manual surveys, enforcement activities, numerical model, etc.)
2. Point of Contact for this Data Management Plan (author or maintainer)
Notes: The name of the Person of the most recent Support Role of type "Metadata Contact" is used. The support role must be in effect.
Notes: The name of the Organization of the most recent Support Role of type "Metadata Contact" is used. This field is required if applicable.
3. Responsible Party for Data Management
Program Managers, or their designee, shall be responsible for assuring the proper management of the data produced by their Program. Please indicate the responsible party below.
Notes: The name of the Person of the most recent Support Role of type "Data Steward" is used. The support role must be in effect.
4. Resources
Programs must identify resources within their own budget for managing the data they produce.
5. Data Lineage and Quality
NOAA has issued Information Quality Guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of information which it disseminates.
(describe or provide URL of description):
Process Steps:
- 2010-01-01 00:00:00 - GPS based surveys were utilized to support both processing and testing of LiDAR data within FEMA designated Areas of Interest (AOIs). Geographically distinct ground points were surveyed using GPS technology throughout the AOIs to provide support for three distinct tasks. Task 1 was to provide Vertical Ground Control to support the aerial acquisition and subsequent bare earth model processing. To accomplish this, survey-grade Trimble R-8 GPS receivers were used to collect a series of control points located on open areas, free of excessive or significant slope, and at least 5 meters away from any significant terrain break. Most if not all control points were collected at street/road intersections on bare level pavement. Task 2 was to collect Fundamental Vertical Accuracy (FVA) checkpoints to evaluate the initial quality of the collected point cloud and to ensure that the collected data was satisfactory for further processing to meet FEMA specifications. The FVA points were collected in identical fashion to the Vertical Ground Control Points, but segregated from the point pool to ensure independent quality testing without prior knowledge of FVA locations by the aerial vendor. Task 3 was to collect Consolidated Vertical Accuracy CVA) checkpoints to allow vertical testing of the bare-earth processed LiDAR data in different classes of land cover, including: Open (pavement, open dirt, short grass), High Grass and Crops, Brush and Low Trees, Forest, Urban. CVA points were collected in similar fashion as Control and FVA points with emphasis on establishing point locations within the predominant land cover classes within each AOI or Functional AOI Group. In order to successfully collect the Forest land cover class, it was necessary to establish a Backsight and Initial Point with the R8 receiver, and then employ a Nikon Total Station to observe a retroreflective prism stationed under tree canopy. This was necessary due to the reduced GPS performance and degradation of signal under tree canopy. The R-8 receivers were equipped with cellular modems to receive real-time correction signals from the Keystone Precision Virtual Reference Station (VRS) network encompassing the Region 1 AOIs. Use of the VRS network allowed rapid collection times (~3 minutes/point) at 2.54 cm (1 inch) initial accuracy. All points collected were below the 8cm specification for testing 24cm, Highest category LiDAR data.
- 2010-01-01 00:00:00 - To ensure valid in-field collections, an NGS monument with suitable vertical reporting was measured using the same equipment and procedures used for Control, FVA and CVA points on a daily basis. The measurement was compared to the NGS published values to ensure that the GPS collection schema was producing valid data and as a physical proof point of quality of collection. Those monument measurements are summarized in the Accuracy report included in the data delivered to FEMA. In order to meet FEMA budgetary requirements, 20 FVA points are necessary to allow testing to CE95 ? 1 point out of 20 may fail vertical testing and still allow the entire dataset to meet 95% accuracy requirements. In similar fashion, 20 CVA points are necessary to test to CE95 as discussed above. 15 CVA points were collected with the intention at the outset that 5 of the collected FVAs would perform double ?duty as Open-class CVA points, to total 20 CVAs. The following software packages and utilities were used to control the GPS receiver in the field during data collection, and then ingest and export the collected GPS data for all points: Trimble Survey Controller, Trimble Pathfinder Office. The following software utilities were used to translate the collected Latitude/Longitude Decimal Degree HAE GPS data for all points into Latitude/Longitude Degrees/Minutes/Seconds for checking the collected monument data against the published NGS Datasheet Lat/Long DMS values and into UTM NAD83 Northings/Eastings: U.S. Army Corps of Engineers CorpsCon, National Geodetic Survey Geoid09NAVD88. MSL values were determined using the most recent NGS-approved geoid model to generate geoid separation values for each Lat/Long coordinate pair. In this fashion, Orthometric heights were determined for each Control, FVA and CVA point by subtracting the generated Geoid Separation value from the Ellipsoidal Height (HAE) for publication and use as MSL NAVD88(09).
- 2010-01-01 00:00:00 - Using an Optech Gemini LiDAR system, a total 111 flightlines of highest density (Nominal pulse Spacing of 1.0m) were collected over the Concord area. A total of 405 square miles was collected. A total of 12 missions were flown between December 2 and December 12, 2010. Two airborne global positioning system (GPS) base stations were used to support the LiDAR data acquisition: BED A-AI5558,and ORH A-AI5600. Coordinates are available in the Post-Flight Aerial Acquisition Report.
- 2011-01-01 00:00:00 - Applanix software was used in the post processing of the airborne GPS and inertial data that is critical to the positioning and orientation of the sensor during all flights. POSPac MMS provides the smoothed best estimate of trajectory (SBET) that is necessary for Optech's post processor to develop the point cloud from the LiDAR missions. The point cloud is the mathematical three dimensional collection of all returns from all laser pulses as determined from the aerial mission. Optech?s DASHMap software and Leica?s ALS Post Processor software were used to create the Raw LIDAR Flight Line strips. At this point this data is ready for analysis, classification, and filtering to generate a bare earth surface model in which the above ground features are removed from the data set. The GeoCue and TerraScan software packages are then used for the automated data classification. Project specific macros are created to classify the ground and to remove the side overlap between parallel flight lines. LAS Class 2 (Ground) is used to check the surveyed control points against the Triangulated LIDAR surface. Any bias is then removed using macro functionality within TerraScan. Unclassified Point Cloud tiles are then created using TerraScan macro functionality. These tiles are populated within GeoCue to ensure correct LAS versioning and LAS Header information. LAS Class 2 is used to check the independent QC points against the Triangulated LiDAR surface. If RMSE is not within guidelines TerraScan software is utilitzed to remove any bias, and the check is performed again.
- 2011-01-01 00:00:00 - Point Cloud data is manually reviewed and any remaining artifacts are removed using functionality provided within the TerraScan and TerraModeler software packages. Additional project specific macros are created and run within GeoCue/TerraScan to ensure correct LAS classification prior to project delivery. Final Classified LAS tiles are created within GeoCue to confirm correct LAS versioning and header information. In-house software is then used to check LAS header information and final LAS classification prior to delivery. LAS Class 2 is used to check the independent QC points against the Triangulated LiDAR surface.
- 2011-01-01 00:00:00 - Created Multipoint feature class from Class 2 LAS files using ArcGIS 3D Analyst converion tool LAS to Multipoint and stored results in APRS_Class_2_Bare_Earth feature dataset within the Concord_HUC8_Bare_Earth_Data.gdb file geodatabase.
- 2011-01-01 00:00:00 - Convert points to raster using the count option...assign one value to all data cells with a conditional statement...fill small nodata areas with expand...reduce the extent with shrink... vectorize with raster to polygon...clean up and create polygon for LiDAR collection area.
- 2011-01-01 00:00:00 - Run point file information tool on classified LAS files...measure tiles and create polyline vector grid that covers collection area...feature to points for the label points of the LAS Information polygon shapefile... feature to polygon using vector grid as polyline input and LAS Info points as label points...clean up file.
- 2011-01-01 00:00:00 - Created ESRI Terrain using class 2 multipoint feature class as mass point input and Concord Collection Area as soft clip. This data is stored in the APRS_Class_2_Bare_Earth feature dataset within the Concord_HUC8_Bare_Earth_Data.gdb file geodatabase.
- 2011-01-01 00:00:00 - Created 5ft floating point raster DEM from ESRI Terrain by converting Terrain to Raster using ArcGIS 3D Analyst. Save results as a ESRI GRID dataset.
- 2011-01-01 00:00:00 - Created 10ft floating point raster DEM from ESRI Terrain by converting Terrain to Raster using ArcGIS 3D Analyst. Save results as a ESRI GRID dataset.
- 2011-01-01 00:00:00 - Create contours by Extracting by mask from the 5ft DEM using a HUC12 area. Save this raster as HUC12 Name 5ft. Focal Statistics using Extracted 5ft DEM as input, Intermediate Focal Raster as Output, Neighborhood should be set to weighted kernel, and the statistic should be sum. Create contours using focal stats raster as input, output polyline should be based on HUC12 name, Contour Interval of 2ft, Set base contour to 0.001. Check results and store in file geodatabase under the HUC12 name feature dataset.
- 2011-01-01 00:00:00 - Create HDEM from the Floating Point 10ft DEM derived from the Concord LiDAR data was constructed to include all drainage areas for the Concord watershed. Used a U.S. Geological Survey (USGS) National Hydrography Dataset (NHD) polyline featureclass containing the stream channel locations was reviewed and compared with orthophotos to confirm the stream channels were in the correct location. Using ArcHydro, the DEM and the polyline shapefile were used as inputs to the DEM Reconditioning (AGREE) tool of Arc Hydro. Arc Hydro burned the polylines into the DEM, and thus produced a hydrologically-correct DEM (HDEM). Sinks are filled, a flow direction and flow accumulation analyses performed, and a final stream GRID developed. The final output of the model is a stream network shapefile. The stream network shapefile is reviewed and compared to orthophotos and USGS NHD flow lines to confirm the stream channels are in the correct location.
- 2013-09-01 00:00:00 - The NOAA Office for Coastal Management (OCM) received the topographic files in LAS V1.2 format. The files contained lidar elevation measurements, classifications, intensity data, return information, GPS time and scan angle. The data were received in NAD83 UTM Zone 19N (meters) and were vertically referenced to NAVD88 using the Geoid09 model. The vertical units of the data were meters. OCM performed the following processing for data storage and Digital Coast provisioning purposes: 1. All points in Class 11 were changed to Class 12 (Overlap) 2. The topographic las files' global encoding bit was set to '1' to reflect use of the Adjusted Standard GPS Time format. 3. The topographic las files were converted from a Projected Coordinate System (NAD83, UTM Zone 19N) to Geographic coordinates (NAD83). 4. The topographic las files' horizontal units were converted from meters to decimal degrees. 5. The topographic las files were converted from orthometric (NAVD88) heights to ellipsoidal heights using Geoid09.
(describe or provide URL of description):
6. Data Documentation
The EDMC Data Documentation Procedural Directive requires that NOAA data be well documented, specifies the use of ISO 19115 and related standards for documentation of new data, and provides links to resources and tools for metadata creation and validation.
Missing/invalid information:
- 1.7. Data collection method(s)
- 3.1. Responsible Party for Data Management
- 4.1. Have resources for management of these data been identified?
- 4.2. Approximate percentage of the budget for these data devoted to data management
- 5.2. Quality control procedures employed
- 7.1. Do these data comply with the Data Access directive?
- 7.1.1. If data are not available or has limitations, has a Waiver been filed?
- 7.1.2. If there are limitations to data access, describe how data are protected
- 7.4. Approximate delay between data collection and dissemination
- 8.1. Actual or planned long-term data archive location
- 8.3. Approximate delay between data collection and submission to an archive facility
- 8.4. How will the data be protected from accidental or malicious modification or deletion prior to receipt by the archive?
(describe or provide URL of description):
7. Data Access
NAO 212-15 states that access to environmental data may only be restricted when distribution is explicitly limited by law, regulation, policy (such as those applicable to personally identifiable information or protected critical infrastructure information or proprietary trade information) or by security requirements. The EDMC Data Access Procedural Directive contains specific guidance, recommends the use of open-standard, interoperable, non-proprietary web services, provides information about resources and tools to enable data access, and includes a Waiver to be submitted to justify any approach other than full, unrestricted public access.
None
Notes: The name of the Organization of the most recent Support Role of type "Distributor" is used. The support role must be in effect. This information is not required if an approved access waiver exists for this data.
Notes: This field is required if a Distributor has not been specified.
https://noaa-nos-coastal-lidar-pds.s3.amazonaws.com/laz/geoid18/2549/index.html
Notes: All URLs listed in the Distribution Info section will be included. This field is required if applicable.
This data can be obtained on-line at the following URL:
https://coast.noaa.gov/dataviewer/#/lidar/search/where:ID=2549
;
Notes: This field is required if applicable.
8. Data Preservation and Protection
The NOAA Procedure for Scientific Records Appraisal and Archive Approval describes how to identify, appraise and decide what scientific records are to be preserved in a NOAA archive.
(Specify NCEI-MD, NCEI-CO, NCEI-NC, NCEI-MS, World Data Center (WDC) facility, Other, To Be Determined, Unable to Archive, or No Archiving Intended)
Notes: This field is required if archive location is World Data Center or Other.
Notes: This field is required if archive location is To Be Determined, Unable to Archive, or No Archiving Intended.
Notes: Physical Location Organization, City and State are required, or a Location Description is required.
Discuss data back-up, disaster recovery/contingency planning, and off-site data storage relevant to the data collection
9. Additional Line Office or Staff Office Questions
Line and Staff Offices may extend this template by inserting additional questions in this section.