Data Management Plan
GUID: gov.noaa.nmfs.inport:55761 | Published / External
Data Management Plan
DMP Template v2.0.1 (2015-01-01)
Please provide the following information, and submit to the NOAA DM Plan Repository.Reference to Master DM Plan (if applicable)
As stated in Section IV, Requirement 1.3, DM Plans may be hierarchical. If this DM Plan inherits provisions from a higher-level DM Plan already submitted to the Repository, then this more-specific Plan only needs to provide information that differs from what was provided in the Master DM Plan.
1. General Description of Data to be Managed
This project merged recently collected topographic, bathymetric, and acoustic elevation data along the entire California coastline from approximately the 10 meter elevation contour out to California's 3 mile state water's boundary. This metadata record describes the Digital Elevation Model (DEM) created from the lidar and multibeam data. The DEM has a 1m cell size. This dataset is has gaps filled by interpolation. See the associated datasets for the version with voids.
Topographic LiDAR: The topographic lidar data used in this merged project was the 2009-2011 CA Coastal Conservancy Lidar Project. The data were collected between October 2009 and August 2011. This collection was a joint effort by the NOAA Office for Coastal Management (OCM); the California State Coastal Conservancy (SCC) Ocean Protection Council (OPC); Scripps Institution of Oceanography; and the Joint Airborne Lidar Bathymetry Technical Center of Expertise (JALBTCX). The data coverage extends landward 500 m from the shoreline, along the entire California coastline. The LAS classifications are as follows: 1-Unclassified, 2-Ground, 7-Noise, 9-Water, 10- Mudflats, 12-Overlap. The LAS points were manually re-classified from water and unclassified to ground in offshore areas where necessary.
Bathymetric LiDAR: The bathymetric lidar data used in this merged project was 2009-2010 U.S. Army Corps of Engineers (USACE) Joint Airborne Lidar Bathymetry Center of Expertise (JALBTCX) lidar, provided by JALBTCX. The data were collected for the California Coastal Mapping Project (CCMP). The original data were in ASCII format and were converted to LAS v1.2. The LAS data were classified as follows: 21-Non-submerged Bathymetry, 22-Bathymetry, 23-Ignored Submerged Bathymetry/Overlap.
Multibeam Acoustic Data: The acoustic data data used in this merged project were provided by the California Seafloor Mapping Program (CSMP) Ocean Protection Council and NOAA's National Geophysical Data Center (NGDC). The original data were in ASCII format and were converted to LAS v1.2. NOAA's VDatum software was used to vertically transform soundings from mean lower low water (MLLW) tidal datum to NAVD88 orthometric datum where necessary. The LAS data were classified as follows: 25-Submerged Acoustic, 26-Ignored Submerged Acoustic/Overlap.
Upon receipt of the data, the NOAA Office for Coastal Management (OCM) converted some of the classifications for data storage and Digital Coast provisioning purposes. The following are the classifications of data available from the NOAA Digital Coast: 1 - Unclassified, 2 - Ground, 7 - Low point (noise), 9 - Water, 11 - Bathymetry, 12 - Overlap, 13 - Submerged Acoustic, 14 - Non-Submerged Bathymetry, 15 - Ignored Submerged Bathymetry/Overlap, 16 - Ignored Submerged Acoustic/Overlap
Notes: Only a maximum of 4000 characters will be included.
Notes: Data collection is considered ongoing if a time frame of type "Continuous" exists.
Notes: All time frames from all extent groups are included.
Notes: All geographic areas from all extent groups are included.
(e.g., digital numeric data, imagery, photographs, video, audio, database, tabular data, etc.)
(e.g., satellite, airplane, unmanned aerial system, radar, weather station, moored buoy, research vessel, autonomous underwater vehicle, animal tagging, manual surveys, enforcement activities, numerical model, etc.)
2. Point of Contact for this Data Management Plan (author or maintainer)
Notes: The name of the Person of the most recent Support Role of type "Metadata Contact" is used. The support role must be in effect.
Notes: The name of the Organization of the most recent Support Role of type "Metadata Contact" is used. This field is required if applicable.
3. Responsible Party for Data Management
Program Managers, or their designee, shall be responsible for assuring the proper management of the data produced by their Program. Please indicate the responsible party below.
Notes: The name of the Person of the most recent Support Role of type "Data Steward" is used. The support role must be in effect.
4. Resources
Programs must identify resources within their own budget for managing the data they produce.
5. Data Lineage and Quality
NOAA has issued Information Quality Guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of information which it disseminates.
(describe or provide URL of description):
Process Steps:
- 2012-12-14 00:00:00 - All data were imported into GeoCue software, converted to LAS v1.2, and transformed, if necessary, to UTM coordinate system, zone 10 North, meters, horizontal datum NAD83 (NSRS2007), vertical datum NAVD88, Geoid 09, meters. Each data type (topographic, bathymetric, and acoustic) were then tiled individually according to project specifications (1500m x 1500m). Data for the NOAA Coastal California Data Merge Project were provided by various sources. Topographic data were provided by NOAA in LAS format, collected for the California Coastal Mapping Project (CCMP). The topographic data were provided with the following classifications: Class 1 = Unclassified (This class includes vegetation, buildings, noise etc.), Class 2 = Ground, Class 7 = Noise, Class 9 = Water, Class 10 = Mudflats, Class 12 = Overlap Bathymetric data were provided by Joint Airborne Bathymetry LiDAR Technical Center of Expertise (JALBTCX) in ASCII format, collected for the National Coastal Mapping Program (NCMP) in 2009. Initial bathymetric data were classified in GeoCue to Class 22 = Submerged bathymetry. Multibeam acoustic data were downloaded from the California Seafloor Mapping Program: Ocean Protection Council (CSMP)(http://seafloor.csumb.edu/SFMLwebDATA.htm), and from NOAA's NGDC (http://maps.ngdc.noaa.gov/viewers/bathymetry/) in ASCII formats, or from USGS open file reports where available. NOAA's VDatum software (version 2.3.5) was used to vertically transform soundings from mean lower low water (MLLW) tidal datum to NAVD88 orthometric datum where necessary.
- 2013-04-12 00:00:00 - Reclassification of Topographic data: Reclassification of offshore islands and pinnacles was a critical component to the success of the final merged product. Tiles within 100m of the coastal shoreline were selected for review and manual classification. Each tile was brought into TerraScan and a temporary surface model was created from the topographic ground points (class 2). Using aerial imagery as a guide, the coastline was examined for any exposed islands that may be incorrectly classified as water in the LAS. Once located, points were reclassified from water (class 9) to ground (class 2) or from unclassified (class 1) to ground. Date for this process step is 20130201-20130412.
- 2013-05-10 00:00:00 - Topographic-Bathymetric Seamline: All breaklines provided by NOAA were merged into a single polygon feature class. This became the foundation of the initial seamline. Areas containing both topographic and bathymetric data along the coastline were reviewed on a tile-by-tile basis using difference rasters. Because the topographic data had better horizontal resolution and vertical accuracy than the bathymetric data, it generally took priority over bathymetric data in overlap areas. However, a smoother transition between the topographic and bathymetric LiDAR datasets may be achieved by modifying the extent of the topographic data in favor of the bathymetric data. In these cases, the breakline was adjusted accordingly. The final polygon feature class was used for reclassification of LAS points in TerraScan. The polygon was converted to a polyline in Arc and clipped to the extent of the coastal shoreline. This final polyline was used as basis for the creation of the DEM smoothing buffers. Date for this process step is 20130222-20130510.
- 2013-02-19 00:00:00 - Acoustic-Acoustic Seamline (Production Block 2b):In surveys with multiple resolutions, the best available resolution data was prioritized and reclassified using TerraScan software. A best fit line was not produced. Between surveys, a best fit line was generated by creating difference rasters and manually digitizing a new boundary.
- 2013-06-21 00:00:00 - Acoustic-Acoustic Seamline (Production Blocks 3 and 4):In bathymetric attributed grid (BAG) surveys with multiple resolutions, the best available resolution data was prioritized and reclassified using TerraScan software. A best fit line was not produced. Between unique surveys, a best fit line was generated by creating difference rasters and manually digitizing a new boundary along areas with minimal offset. Date for this process setp 20130501-20130621.
- 2013-02-27 00:00:00 - Bathymetric-Acoustic Seamline (Production Block 2b): The bathymetric-acoustic seamline was generated using the same methodology as the acoustic overlap seamline. The final polyline was used as basis for the creation of the DEM smoothing buffers.
- 2013-06-21 00:00:00 - Bathymetric-Acoustic Seamline (Production Blocks 3 and 4):The bathymetric-acoustic seamline was generated using the same methodology as the acoustic overlap seamline, using difference rasters and manually digitizing seamlines with minimal offset. The final polyline was used as basis for the creation of the DEM smoothing buffers. The date for this process step is 20130501-20130621.
- 2013-05-17 00:00:00 - Reclassification of Bathymetric data: In areas of overlap where topographic data were given priority over bathymetric data, overlapping bathymetric points were moved to class 21(non-submerged bathymetry). This was performed in TerraScan using the final seamline polygon. A minimum of three (3) data points within a 15m eter radius is required for inclusion in the DEM. An extent polygon is needed to accomplish this task. Topographic ground points (class 2) and bathymetric points (class 22) were converted to a masspoint feature class in Arc. The masspoints were used as the input in the Aggregate Points tool in Arc with a set distance of 15m. After the final extent polygons were generated, class 22 bathymetric points not within these polygons were moved to class 23 in TerraScan. Points outside the extent polygon remained in class 21/22. Finally, to remove above-ground artifacts from the bathymetric data, all class 22 points with elevations above +1m were reassigned to class 21 using an automated process in TerraScan. This removed docks, piers, bridges, and other miscellaneous artifacts from the point cloud. Bathymetric data between 0-1m elevation were left in class 22 to avoid classifying out too many data points in the nearshore areas. Bathymetric elevations between 0-1m were evaluated on a tile-by-tile basis and moved to class 21 if necessary. The date for this process step is 20130304-20130517.
- 2013-03-06 00:00:00 - Reclassification of Acoustic data (Production Block 2b):Each area of the acoustic data was reclassified from the temporary class codes to the final class 25 (submerged multibeam acoustic) and class 26 (acoustic overlap) designations using the polygons generated from the acoustic-acoustic seamline boundaries. Best resolution data was prioritized in multi-resolution surveys. Best fit seamline was used between surveys.
- 2013-06-21 00:00:00 - Reclassification of Acoustic data (Production Blocks 3 and 4):Each area of the acoustic data was reclassified from the temporary class codes to the final class 25 (submerged multibeam acoustic) and class 26 (acoustic overlap) designations. Voids in high resolution acoustic BAG data were filled with available lower resolution BAG data for each acoustic data set. This is an automated process using Fugro proprietary software specially developed for this project. This process results in a merged LAS point cloud for each survey location utilizing the best resolution BAG data. The ignored points were classified as class 26. The same filling process was used in case of NCCMP data for survey locations where data of various acquisition dates were available within a survey. Priority was given to data collected in 2010, following by 2008 and older datasets. Surface subtraction images were generated and utilized to digitize best fit line between overlapping acoustic project data sets. Overlapping acoustic datasets were reclassified and merged based on the best fit line using LAS mosaic software developed by Fugro. The merged point cloud data for each acoustic project was tiled to the project required tiling scheme. The date for this process step is 20130501-20130621.
- 2013-03-06 00:00:00 - Reclassification of Bathymetric data (Production Block 2b):In overlapping areas between acoustic and bathymetric datasets, priority was given to the acoustic dataset due to more efficient coverage and higher resolution where available. With exception of two instances where best fit line was used, the bathymetric-acoustic seamline was represented by geographic extent of class 25. The bathymetric data within this extent was reclassified to class 23.
- 2013-06-21 00:00:00 - Reclassification of Bathymetric data (Production Blocks 3 and 4):In overlapping areas between acoustic and bathymetric datasets, priority was given to the acoustic dataset due to more efficient coverage and higher resolution where available. With exception of two instances where best fit line was used, the bathymetric-acoustic seamline was represented by geographic extent of class 25. The bathymetric data within this extent was reclassified to class 23. Acoustic point cloud and bathymetric lidar data were merged using TSCAN project. The date for this process step is 20130501-20130621.
- 2013-01-17 00:00:00 - Topographic-Bathymetric-Acoustic data merge process: Class 2, topographic ground, class 22, bathymetric ground, and class 25, acoustic ground, LiDAR points were exported from the combined LAS files into an Arc Geodatabase (GDB) in multipoint format. An ESRI Terrain was generated using 0.7 average point spacing. The ESRI Terrain was converted to an ERDAS Imagine raster with a 1 meter cell size using the natural neighbors interpolation method. The resulting raster was smoothed to reduce vertical offsets between the topographic-bathymetric and bathymetric-acoustic datasets. The best fit seamlines were used as the starting point for the smoothing process of the DEM. Using the raster as a visual guide, the seamlines were reviewed manually on a tile by tile basis in Global Mapper. The lines were clipped and removed in areas where vertical offests were not visible in the DEM or where cliffs, berms, voids, or other features would be not be smoothed. This prevents any unnecessary alteration of the original elevation data. A 10m radius buffer polygon was generated in Arc around the remaining portions of the best fit line. In Arc, the polygon buffer was used to extract a raster "strip" that would be used in the smoothing process. After extraction, the cell values in this raster strip were resampled to 3m cell size using mean aggregation. The resulting raster strip was resampled back to a 1m cell size using bilinear interpolation. This process produced a raster strip that would be used to smooth between data sources. The raster, the raster strip, and the seamline polygon buffer were opened in Global Mapper software. To ensure that raster values are not automatically resampled in Global Mapper when performing the final export, the default raster interpolation option was changed to Nearest Neighbor (none) before any further processing was performed. The seamline polygon buffer was selected in Global Mapper and the raster strip was feathered by 8 pixels. This eliminated any hard edges that would normally be produced by merging the two raster datasets. The two rasters were exported as one merged raster from Global Mapper using the following parameters: File Type: ERDAS Imagine (IMG) Cell Size: 1x1 m Data: 32-bit floating elevation Create Compressed File (UNCHECKED) Interpolate Gaps (CHECKED) The DEM was reopened in ESRI ArcGIS software and QCed. Any alignment shifts, projection issues, or other problems were corrected as necessary. This satisfies the Smoothed with Interpolated Voids portion of the project. To produce the Smoothed with Voids DEM, a void mask was first generated from the masspoints using the point aggregate tool in Arc. Aggregation distance was set to 15m. Extents were generated past the project boundary to prevent excessive voids along the edges. The resulting data extent was buffered by 1m. This prevents points on the edge of the extents from being included in the final void mask. The buffered extent was inverted using the symmetrical difference tool in Arc. These areas represent the new initial data voids layer. Multipart features were exploded and area calculations were made on the resulting void polygons. Void polygons smaller than 225 sq meters in area were removed. Remaining void polygons within the topographic area not within breakline boundaries were also removed. Inland breaklines were converted to polygons and added to the void mask as necessary. The void mask was converted to raster format and used to remove voids from the smoothed raster. This satisfies the Smoothed with Voids portion of the project. Each DEM was clipped to the project boundary.The date for this process step is 20130420-20130717.
- 2013-07-19 00:00:00 - Each DEM was clipped to individual tiles. Dewberry uses a proprietary tool that clips the area DEMs to each tile located within the final Tile Grid, names the clipped DEM to the Tile Grid Cell name, and verifies that final extents are correct. All individual tiles were loaded into Global Mapper for the last review. During this last review, products are checked to ensure full, complete coverage, no issues along tile boundaries, tiles seamlessly edge-match, and that there are no remaining processing artifacts in the dataset.The date for this process step is 20130422-20130719
(describe or provide URL of description):
6. Data Documentation
The EDMC Data Documentation Procedural Directive requires that NOAA data be well documented, specifies the use of ISO 19115 and related standards for documentation of new data, and provides links to resources and tools for metadata creation and validation.
Missing/invalid information:
- 1.6. Type(s) of data
- 1.7. Data collection method(s)
- 3.1. Responsible Party for Data Management
- 4.1. Have resources for management of these data been identified?
- 4.2. Approximate percentage of the budget for these data devoted to data management
- 5.2. Quality control procedures employed
- 7.1. Do these data comply with the Data Access directive?
- 7.1.1. If data are not available or has limitations, has a Waiver been filed?
- 7.1.2. If there are limitations to data access, describe how data are protected
- 7.4. Approximate delay between data collection and dissemination
- 8.1. Actual or planned long-term data archive location
- 8.3. Approximate delay between data collection and submission to an archive facility
- 8.4. How will the data be protected from accidental or malicious modification or deletion prior to receipt by the archive?
(describe or provide URL of description):
7. Data Access
NAO 212-15 states that access to environmental data may only be restricted when distribution is explicitly limited by law, regulation, policy (such as those applicable to personally identifiable information or protected critical infrastructure information or proprietary trade information) or by security requirements. The EDMC Data Access Procedural Directive contains specific guidance, recommends the use of open-standard, interoperable, non-proprietary web services, provides information about resources and tools to enable data access, and includes a Waiver to be submitted to justify any approach other than full, unrestricted public access.
None
Notes: The name of the Organization of the most recent Support Role of type "Distributor" is used. The support role must be in effect. This information is not required if an approved access waiver exists for this data.
Notes: This field is required if a Distributor has not been specified.
https://noaa-nos-coastal-lidar-pds.s3.us-east-1.amazonaws.com/dem/California_Topobathy_DEM_Interp_8684/index.html
Notes: All URLs listed in the Distribution Info section will be included. This field is required if applicable.
This data can be obtained on-line at the following URL:https://coast.noaa.gov/dataviewer/#/lidar/search/where:ID=2616 This data set is dynamically generated based on user-specified parameters.;
Notes: This field is required if applicable.
8. Data Preservation and Protection
The NOAA Procedure for Scientific Records Appraisal and Archive Approval describes how to identify, appraise and decide what scientific records are to be preserved in a NOAA archive.
(Specify NCEI-MD, NCEI-CO, NCEI-NC, NCEI-MS, World Data Center (WDC) facility, Other, To Be Determined, Unable to Archive, or No Archiving Intended)
Notes: This field is required if archive location is World Data Center or Other.
Notes: This field is required if archive location is To Be Determined, Unable to Archive, or No Archiving Intended.
Notes: Physical Location Organization, City and State are required, or a Location Description is required.
Discuss data back-up, disaster recovery/contingency planning, and off-site data storage relevant to the data collection
9. Additional Line Office or Staff Office Questions
Line and Staff Offices may extend this template by inserting additional questions in this section.