Data Management Plan (Deprecated)

Axis Geospatial has contracted Atlantic to obtain professional services related to the development of Light Detection and Ranging (LiDAR) data between April 2015 and June 2015. This data set is consisting of LiDAR Point Cloud, Classified LiDAR, Digital Elevation Model, and LiDAR Intensity Images which all pieces encompasses the entire 2,424 Square miles (6,278 km) of the NY_WarrenWashingtonEssex_Spring2015 project area. The project area spans 5 counties including Warren, Washington, Essex, Hamilton, and Franklin. This data set consists of LiDAR point cloud with nominal pulse spacing of 0.5557 m, and tiled Las files. Each LAS file contains LiDAR point information, which has been calibrated, controlled, and classified.

Ground Conditions: water at normal levels; no unusual inundation; no snow; leaf off

The calibrated point cloud data from the laser sensor was merged to produce processed (*.las) files including but not limited to 3D position, intensity, and time-stamp. A filtering methodology was utilized to produce a multi-return surface elevation model dataset with bare-earth conditions. GeoCue, TerraScan, and TerraModel software was used for the initial batch processing and manual editing of the (*.las) point clouds. The errors (noise columns) in classification were corrected by identifying the areas of dense noise in the point cloud and manually removing the incorrectly classified ground surfaces and above ground features.The Withheld and Overlap flags were utilized for the final LAS v1.4 file production. These classified LiDAR points were then used in the creation of other deliverables such as Bare-Earth Digital Elevation Models (DEMs) and LiDAR Intensity images. The final deliverables include 2783 tiles.

How the Withheld Points are Identified: Withheld (ignore) points were identified in the files using the standard LAS Withheld bit.

Class Code:0

Class Item: Never Classified

Class Code:1

Class Item: Undetermined/Unclassified

Class Code:2

Class Item: Ground

Class Code:7

Class Item:Low Point (Noise)

Class Code:17

Class Item:Bridge Deck

Class Code:18

Class Item:High Noise

In addition to these lidar point data, the bare earth Digital Elevation Models (DEM) created from the lidar point data are also available. These data are available for custom download at the link provided in the URL section of this metadata record.

Lineage Statement:
Data were collected and processed by Atlantic for the New York State GIS Program Office (NYSGPO) and posted to their ftp site. The NOAA Office for Coastal Management (OCM) downloaded the data from the ftp site and processed it to make it available for custom download from the Data Access Viewer (DAV) and bulk download from https.

Process Steps:

• 2015-05-23 00:00:00 - Aerial LiDAR Acquisition: Aerial data collection was acquired in sixteen (16) missions, using the ALS70 SN#7123 at a max flying height of 3500 AGL. This was to support a 2.5 ppm^2 LiDAR point cloud. Airborne GPS and IMU data was collected during the acquisition and supported by Topcon Viper GPS, Data acquisition was completed on May 23rd 2015. Ground Control Survey: A survey was performed by Axis Geospatial to support the acquisition of Light Detection and Ranging (LiDAR). The control network involved a total of 673 check points (294 NVA + 379 VVA). The points were a combination of the following ground cover classification: Open Terrain, Urban Terrain, Bare Earth, and High Grass.
• 2015-06-01 00:00:00 - LiDAR Pre-processing: Airborne GPS and IMU data were merged to develop a Single Best Estimate (SBET) of the LiDAR system trajectory for each lift. LiDAR ranging data were initially calibrated using previous best parameters for this instrument and aircraft. Relative calibration was evaluated using advanced plane-matching analysis and parameter corrections derived. This process was repeated interactively until residual errors between overlapping swaths, across all project lifts, was reduced to 2 cm or less. Data was then block adjusted to match surveyed calibration control. Raw data NVA was checked using independently surveyed check points. Swath overage points were identified and tagged within each swath file.
• 2015-06-15 00:00:00 - Noise Columns: The isolated dense erroneous points presenting as columns of noise both above and below the bare-earth surface are almost entirely in Blocks 1 and 2 (Affected Tiles shapefile attached). This type of noise is only present in the SN7123 ALS70 sensor acquired data in the southern half of the project area. A combination of the acquisition discriminator settings, flying height, FOV, terrain, and surface reflectance were the cause of these occurrences. In these instances the scanner’s nadir position (scanner nadir as opposed to the center of the acquired swath) in correlation to the planar surface of a roof or waterbody exhibited a bias in the digitized intensity from the low-gain channels. As the laser output is scanned through the point where the laser output is perpendicular to the roof or water surface with very low reflectance the increased sensitivity from the acquisition discriminator settings resulted in an increase of laser backscatter reflected into the receiver. That backscatter was represented as columns of erroneous points or noise, while this did not affect the laser returns at ground level it did effect the automated high noise filtering in the initial classification process.
• 2015-06-01 00:00:00 - Initial processing of the GPS data was processed using Inertial Explorer. The solution file was generated and CloudPro software was used to generate georeferenced laser returns which were then processed in strip form allowing for the QC of the overlap between strips (lines). The data from each line were combined and automated classification routines run to determine the initial surface model. This initial surface model was then verified to the surveyed test points.
• 2015-06-10 00:00:00 - LiDAR Post-Processing: The calibrated and controlled LiDAR swaths were processed using automatic point classification routines in TerraSolid software. These routines operate against the entire collection (all swaths, all lifts), eliminating character differences between files. Data were then distributed as virtual tiles to experienced LiDAR analysts for localized automatic classification, manual editing, and peer-based QC checks. Supervisory QC monitoring of work in progress and completed editing ensured consistency of classification character and adherence to project requirements across the entire project. All classification tags are stored in the original swath files. After completion of classification and final QC approval, the NVA and VVA for the project are calculated. Sample areas for each land cover type present in the project were extracted and forwarded to the client, along with the results of the accuracy tests. Upon acceptance, the complete classified LiDAR swath files were delivered to the client.
• 2015-07-01 00:00:00 - LiDAR Classification: The calibrated LiDAR data was run through automated classification routines and then manually checked and edited. The data was classified into the following classes: 1-unclassified*, 2-ground, 7-low noise, 17-bridges, and 18-high noise
• 2015-07-01 00:00:00 - LiDAR Intensity Imagery Creation: All LiDAR Intensity Imagery was created from the final calibrated and classified LiDAR Point Cloud. Intensity Images where produced from all classified points and are posted to a 1.0 meter cell size. Intensity Images where cut to match the provided Tile Index and have corresponding names to match tile names.
• 2015-07-10 00:00:00 - Bare-Earth DEM Creation: Bare-Earth Digital Elevation Models (DEMs) were derived using bare-earth (ground) LiDAR points, all DEMs were created with a grid spacing of 1 meter. The DEMs were cut to tiles of 1500 meters X 1500 meters index, provided by Axis Geospatial.
• 2021-08-09 00:00:00 - The NOAA Office for Coastal Management (OCM) downloaded 2782 las files from ftp://ftp.gis.ny.gov/elevation/LIDAR/NYSGPO_WarrenWashingtonEssex_2015/. The data were in UTM Zone 18 (NAD83 2011), meters coordinates and NAVD88 (Geoid12A) elevations in meters. The data were classified as: 1 - Unclassified, 2 - Ground, 7 - Low Noise, 9 - Water, 17 - Bridge Decks, 18 - High Noise. OCM processed all classifications of points to the Digital Coast Data Access Viewer (DAV). Classes available on the DAV are: 1, 2, 7, 9, 17, 18. Although not listed in the original contractor metadata, points classified as 9 - water are included. There were also a relatively low number of points that were classified with random classes. These values were for the most part very high or very low (blunder) elevations. These relatively few points with random classifications remain in the data set. OCM performed the following processing on the data for Digital Coast storage and provisioning purposes: 1. The las files were converted from las format to laz format using laszip. 2. Internal OCM scripts were run to check the number of points by classification and by flight ID and the gps, elevation, and intensity ranges. 3. Internal OCM scripts were run on the laz files to: a. Convert from orthometric (NAVD88) elevations to ellipsoid elevations using the Geoid12A model b. Convert the laz files from UTM Zone 18 (NAD83 2011), meters coordinates to geographic coordinates c. Filter points with elevation values less than 0 meters and greater than 2500 meters. d. Filter points outside the geographic range of the bounding coordinates of -81, 40, -72, 45.5 degrees e. The filtering indicated that four files had points that were clipped out. Those files were: u_6300085100_2015, u_6165084650_2015, u_6120081050_2015, u_5745085100_2015 f. Assign the geokeys, sort the data by gps time and zip the data to database and to http.

Data is available online for bulk and custom downloads.

Data is backed up to tape and to cloud storage.