Data Management Plan

Original Dataset Product: These are Digital Elevation Model (DEM) data for California as part of the required deliverables for the CA_SanFrancisco_B23, Work Unit 300449 project. Class 2 (ground) LiDAR points in conjunction with the hydro breaklines were used to create a 0.25 meter hydro-flattened Raster DEM.

Original Dataset Geographic Extent: 4 counties (Alameda, Marin, San Francisco, San Mateo) in California, covering approximately 53 total square miles.

Original Dataset Description: The CA_SanFrancisco_B23, Work Unit 300449 project called for the planning, acquisition, processing, and derivative products of lidar data to be collected at a nominal pulse spacing (NPS) of 0.15 meters. Project specifications are based on the U.S. Geological Survey National Geospatial Program Base LiDAR Specification, 2023 Rev. A. The data were developed based on a horizontal projection/datum of NAD 1983 2011 San Francisco CS13 Meters, Meter and vertical datum of NAVD88 Geoid18, Meter. LiDAR data were delivered as processed Classified LAS 1.4 files formatted to 654 individual 1000 m x 1000 m tiles, as tiled intensity imagery, and as tiled bare earth DEMs; all tiled to the same 1000 m x 1000 m schema. Continuous breaklines were produced in Esri file geodatabase format.

Original Dataset Collection Ground Conditions: LiDAR was collected in spring 2023, while no snow was on the ground and rivers were at or below normal levels. In order to post process the LiDAR data to meet task order specifications and meet ASPRS vertical accuracy guidelines, NV5 Geospatial utilized a total of 13 ground control points that were used to calibrate the LiDAR to known ground locations established throughout the project area. An additional 41 independent accuracy checkpoints, 30 in Bare Earth and Urban landcovers (30 NVA points), 11 in Tall Weeds categories (11 VVA points), were used to assess the vertical accuracy of the data. These checkpoints were not used to calibrate or post process the data.

This metadata record supports the data entry in the NOAA Digital Coast Data Access Viewer (DAV). For this data set, the DAV is leveraging the USGS GeoTiff files hosted by USGS on Amazon Web Services.

Lineage Statement:
The NOAA Office for Coastal Management (OCM) ingested references to the USGS GeoTiff files that are hosted on Amazon Web Services (AWS), into the Digital Coast Data Access Viewer (DAV). The DAV accesses the raster data as it resides on AWS under the usgs-lidar-public-container.

Process Steps:

• 2024-01-01 00:00:00 - Raw Data and Boresight Processing: The boresight for each lift was done individually as the solution may change slightly from lift to lift. The following steps describe the Raw Data Processing and Boresight process: 1) Technicians processed the raw data to LAS format flight lines using the final GPS/IMU solution. This LAS data set was used as source data for boresight. 2) Technicians first used NV5 Geospatial proprietary and commercial software to calculate initial boresight adjustment angles based on sample areas selected in the lift. These areas cover calibration flight lines collected in the lift, cross tie, and production flight lines. These areas are well distributed in the lift coverage and cover multiple terrain types that are necessary for boresight angle calculation. The technicians then analyzed the results and made any necessary additional adjustment until it was acceptable for the selected areas. 3) Once the boresight angle calculation was completed for the selected areas, the adjusted settings were applied to all of the flight lines of the lift and checked for consistency. The technicians utilized commercial and proprietary software packages to analyze how well flight line overlaps matched for the entire lift and adjusted as necessary until the results met the project specifications. 4) Once all lifts were completed with individual boresight adjustment, the technicians checked and corrected the vertical misalignment of all flight lines and also the matching between data and ground truth. The relative accuracy was less than or equal to 7 cm RMSEz within individual swaths and less than or equal to 10 cm RMSEz or within swath overlap (between adjacent swaths). 5) The technicians ran a final vertical accuracy check of the boresighted flight lines against the surveyed checkpoints after the z correction to ensure the requirement of NVA = 19.6 cm 95% Confidence Level (Required Accuracy) was met.
• 2024-01-01 00:00:00 - LAS Point Classification: The point classification is performed as described below. The bare earth surface is then manually reviewed to ensure correct classification on the Class 2 (Ground) points. After the bare-earth surface is finalized, it is then used to generate all hydro-breaklines through heads-up digitization. All ground (ASPRS Class 2) lidar data inside of the Lake Pond and Double Line Drain hydro flattening breaklines were then classified to water (ASPRS Class 9) using TerraScan macro functionality. A buffer of 0.5 feet was also used around each hydro-flattened feature to classify these ground (ASPRS Class 2) points to Ignored ground (ASPRS Class 10). All Lake Pond Island and Double Line Drain Island features were checked to ensure that the ground (ASPRS Class 2) points were reclassified to the correct classification after the automated classification was completed. All overlap data was processed through automated functionality provided by TerraScan to classify the overlapping flight line data to approved classes by USGS. The overlap data was classified using standard LAS overlap bit. These classes were created through automated processes only and were not verified for classification accuracy. Due to software limitations within TerraScan, these classes were used to trip the withheld bit within various software packages. These processes were reviewed and accepted by USGS through numerous conference calls and pilot study areas. All data was manually reviewed and any remaining artifacts removed using functionality provided by TerraScan and TerraModeler. Global Mapper us used as a final check of the bare earth dataset. GeoCue was then used to create the deliverable industry-standard LAS files for both the All Point Cloud Data and the Bare Earth. Proprietary software was used to perform final statistical analysis of the classes in the LAS files, on a per tile level to verify final classification metrics and full LAS header information.
• 2024-01-01 00:00:00 - Maximum Surface Height Rasters were produced as ancillary data from the classified lidar point cloud in order to evaluate the withheld bit flag proof of performance for points that cannot be reasonably interpreted as valid surface returns. These were produced using all returns, withheld flagged points excluded, and using the highest elevation point value from each pixel. These rasters are 32-bit, floating point format and delivered as GeoTIFF files per tile. MSHR are generated from the point cloud data and will not be altered after creation nor will there be further maintenance on this product.
• 2024-01-01 00:00:00 - Swath Separation Imagery was produced for the entire project area. Swath separation images use color-coding to illustrate differences in elevation (z-) values where swaths overlap. The color-coded images are semi-transparent and overlay the lidar intensity image. They are ancillary data used as visual aids to more easily identify regions within point cloud datasets that may have suspect interswath alignment or other geometric issues. Imagery was created using last returns with all classification and bit flags, except for noise and withheld bit flag are included. SSI are generated from the point cloud data and will not be altered after creation nor will there be further maintenance on this product.
• Hydro-Flattened Raster DEM Processing: Class 2 (Ground) LiDAR points in conjunction with the hydro-breaklines were used to create a 0.25 meter hydro-flattened raster DEM. Using automated scripting routines in proprietary software, GeoTIFF files were created for each tile. Each surface was reviewed using Global Mapper to check for any surface anomalies or incorrect elevations found within the surface.
• 2024-09-16 00:00:00 - The NOAA Office for Coastal Management (OCM) created references to the USGS GeoTiff files that were ingested into the NOAA Digital Coast Data Access Viewer (DAV). No changes were made to the data. The DAV will access the raster data as it resides on Amazon Web Services (AWS) under the usgs-lidar-public container. This is the location of the GeoTiffs that are being accessed: https://prd-tnm.s3.amazonaws.com/index.html?prefix=StagedProducts/Elevation/OPR/Projects/CA_SanFrancisco_B23/CA_SanFrancisco_1_B23/TIFF/

Data is available online for bulk and custom downloads.

Data is backed up to cloud storage.