Slide Menu
Search Help Show/Hide Menu
Short Citation:
OCM Partners, 2022: 2004 USGS Lidar: San Francisco Bay (CA), https://www.fisheries.noaa.gov/inport/item/49635.

Item Identification

Title: 2004 USGS Lidar: San Francisco Bay (CA)
Short Name: ca2004_usgs_sanfranbay_m5009_metadata
Status: Completed
Publication Date: 2016-03-11
Abstract:

Lidar (Light detection and ranging) discrete-return point cloud data are available in the American Society for Photogrammetry and Remote Sensing (ASPRS) LAS format. The LAS format is a standardized binary format for storing 3-dimensional point cloud data and point attributes along with header information and variable length records specific to the data. Millions of data points are stored as a 3-dimensional data cloud as a series of x (longitude), y (latitude) and z (elevation) points.

Purpose:

High-resolution digital elevation maps generated by airborne and stationary LiDAR have led to significant advances in geomorphology, the branch of geoscience concerned with the origin and evolution of Earth's surface topography. LiDAR provides unique characteristics relative to other remotely sensed data sources by providing three-dimensional feature information that cannot be derived from traditional imaging sensors.

Notes:

10279

Supplemental Information:

A report for this project is available at:

https://noaa-nos-coastal-lidar-pds.s3.amazonaws.com/laz/geoid18/5009/supplemental/ca2004_usgs_sanfranbay_m5009_lidarreport.pdf

A footprint of this data set may be viewed in Google Earth at:

https://noaa-nos-coastal-lidar-pds.s3.amazonaws.com/laz/geoid18/5009/supplemental/ca2004_usgs_sanfranbay_m5009.kmz

Keywords

Theme Keywords

Thesaurus Keyword
ISO 19115 Topic Category
elevation

Physical Location

Organization: Office for Coastal Management
City: Charleston
State/Province: SC

Data Set Information

Data Set Scope Code: Data Set
Maintenance Frequency: As Needed
Entity Attribute Overview:

LAS 1.2 format (classes 1,2,4,9)

Entity Attribute Detail Citation:

none

Distribution Liability:

Any conclusions drawn from the analysis of this information are not the responsibility of SFSU, USGS, NOAA, the Office for Coastal Management or its partners.

Support Roles

Data Steward

CC ID: 670724
Date Effective From: 2016-03-11
Date Effective To:
Contact (Organization): NOAA Office for Coastal Management (NOAA/OCM)
Address: 2234 South Hobson Ave
Charleston, SC 29405-2413
Email Address: coastal.info@noaa.gov
Phone: (843) 740-1202
URL: https://coast.noaa.gov

Distributor

CC ID: 670726
Date Effective From: 2016-03-11
Date Effective To:
Contact (Organization): NOAA Office for Coastal Management (NOAA/OCM)
Address: 2234 South Hobson Ave
Charleston, SC 29405-2413
Email Address: coastal.info@noaa.gov
Phone: (843) 740-1202
URL: https://coast.noaa.gov

Metadata Contact

CC ID: 670727
Date Effective From: 2016-03-11
Date Effective To:
Contact (Organization): NOAA Office for Coastal Management (NOAA/OCM)
Address: 2234 South Hobson Ave
Charleston, SC 29405-2413
Email Address: coastal.info@noaa.gov
Phone: (843) 740-1202
URL: https://coast.noaa.gov

Point of Contact

CC ID: 670725
Date Effective From: 2016-03-11
Date Effective To:
Contact (Organization): NOAA Office for Coastal Management (NOAA/OCM)
Address: 2234 South Hobson Ave
Charleston, SC 29405-2413
Email Address: coastal.info@noaa.gov
Phone: (843) 740-1202
URL: https://coast.noaa.gov

Extents

Currentness Reference: Ground Condition

Extent Group 1

Extent Group 1 / Geographic Area 1

CC ID: 1137622
W° Bound: -122.1251999
E° Bound: -122.0622972
N° Bound: 37.437986
S° Bound: 37.4205777

Extent Group 1 / Time Frame 1

CC ID: 1137621
Time Frame Type: Range
Start: 2004-05-05
End: 2004-05-21

Spatial Information

Spatial Representation

Representations Used

Vector: Yes

Access Information

Security Class: Unclassified
Data Access Procedure:

This data can be obtained on-line at the following URL:

https://coast.noaa.gov/dataviewer/#/lidar/search/where:ID=5009

The data set is dynamically generated based on user-specified parameters.;

Data Access Constraints:

None

Data Use Constraints:

Users should be aware that temporal changes may have occurred since this data set was collected and some parts of this data may no longer represent actual surface conditions. Users should not use this data for critical applications without a full awareness of its limitations. These data depict the heights at the time of the survey and are only accurate for that time.

Distribution Information

Distribution 1

CC ID: 740459
Download URL: https://coast.noaa.gov/dataviewer/#/lidar/search/where:ID=5009
Distributor:
File Name: Customized Download
Description:

Create custom data files by choosing data area, product type, map projection, file format, datum, etc.

Distribution 2

CC ID: 740460
Download URL: https://noaa-nos-coastal-lidar-pds.s3.amazonaws.com/laz/geoid18/5009/index.html
Distributor:
File Name: Bulk Download
Description:

Simple download of data files.

URLs

URL 1

CC ID: 740462
URL: https://coast.noaa.gov/dataviewer
URL Type:
Online Resource

URL 2

CC ID: 740463
URL: https://coast.noaa.gov
URL Type:
Online Resource

URL 3

CC ID: 740464
URL: https://noaa-nos-coastal-lidar-pds.s3.amazonaws.com/laz/geoid18/5009/supplemental/ca2004_usgs_sanfranbay_m5009.kmz
Name: Browse Graphic
URL Type:
Browse Graphic
File Resource Format: kmz
Description:

This graphic shows the lidar coverage for the 2004 USGS project along the southern area of San Francisco Bay.

Activity Log

Activity Log 1

CC ID: 670746
Activity Date/Time: 2017-03-20
Description:

Date that the source FGDC record was last modified.

Activity Log 2

CC ID: 670745
Activity Date/Time: 2017-11-14
Description:

Converted from FGDC Content Standards for Digital Geospatial Metadata (version FGDC-STD-001-1998) using 'fgdc_to_inport_xml.pl' script. Contact Tyler Christensen (NOS) for details.

Activity Log 3

CC ID: 718177
Activity Date/Time: 2018-02-08
Description:

Partial upload of Positional Accuracy fields only.

Activity Log 4

CC ID: 740461
Activity Date/Time: 2018-03-13
Description:

Partial upload to move data access links to Distribution Info.

Data Quality

Accuracy:

Data are provided in las format version las 1.0. Header attributes and point attribute for LAS files can be found on ASPRS website listed below. USGS not making any promises on the accuracy or completeness of the attribute information.

Horizontal Positional Accuracy:

Accuracy is as follows, quoted at the 95% confidence level (2 sigma); Quantitative Value: 0.40 meters, Test that produced the value: +/- 20-60 cm on all but extremely hilly terrain

Vertical Positional Accuracy:

Accuracy is as follows, quoted at the 95% confidence level (2 sigma); Quantitative Value: 0.0647 meters, Test that produced the value: RMSE (1 sigma) calculated to be 6.47 cm. 593 kinematic check points compared to the LiDAR bald earth surface on paved surfaces found that the average error was -1.9 centimeters and that 95% (2 sigma) of the checkpoints were within 13.2 centimeters of true

values.

Completeness Report:

The lidar collection has been acquired by the USGS through contracts, partnerships with other Federal, state, tribal, or regional agencies, from direct purchases from private industry vendors, and through volunteer contributions from the science community. While USGS makes every effort to provide accurate and complete information, USGS provides no warranty, expressed or implied, as to the accuracy, reliability or completeness of furnished lidar point clouds. Please note that USGS does not control and cannot guarantee the relevance, timeliness, or accuracy of these outside materials.

Conceptual Consistency:

Please refer to the Metadata Section at the end of this document for a live link to a copy of the original data source's metadata file if one was provided.

Lineage

Process Steps

Process Step 1

CC ID: 1137615
Description:

The points are generated as Terrascan binary Format using Terrapoint's proprietary Laser Postprocessor Software. This software combines the Raw Laser file and GPS/IMU information to generate a point cloud for each individual flight. All the point cloud files encompassing the project area were then divided into 2 kilometer by 2-kilometer tiles. The referencing system of these tiles is based upon the project boundary minimum and maximums. This process is carried out in Terrascan.

The bald earth is subsequently extracted from the raw LiDAR points using Terrascan in a Microstation environment. The automated vegetation removal process takes place by building an iterative surface model. This surface model is generated using three main parameters: Building size, Iteration angle and Iteration distance. The initial model is based upon low points selected by a roaming window and are assumed to be ground points. The size of this roaming window is determined by the building size parameter. These low points are triangulated and the remaining points are evaluated and subsequently added to the model if they meet the Iteration angle and distance constraints (fig. 1). This process is repeated until no additional points are added within an iteration. There is also a maximum terrain angle constraint that determines the maximum terrain angle allowed within the model.

Quality Control

Once the data setup has taken place the manual quality control of the surface occurs. This process consists of visually examining the LiDAR points within Terrascan and correcting errors that occurred during the automated process. These corrections include verifying that all non ground elements, such as vegetation and buildings are removed from the ground model and that all small terrain undulations such as road beds, dykes, rock cuts and hill tops are present within the model. This process is done with the help of hillshades, contours, profiles and crosssections. To correct misclassifications, a full suite of Terrascan and custom in-house data tools are used.

Deliverables

All LiDAR Data Products were delivered on DVD-ROM. Three copies were provided. All products other than hill-shade data were provided in 1k and 2k tiles with a 25-meter buffer. Hill-shades were delivered in three large areas. Full Feature or All Return Point Data Data delivered in ASCII, comma delimited files with one record per return as defined in Table 4. The records are ordered sequentially according to Easting with no duplicate records. The individual returns are classified into the categories as defined in Table 5. If a return cannot be reliably classified as "Vegetation" or "Building/structure", it shall be assigned as "Not ground". This product was generated using an in house custom utility. The process involved extracting a Terrascan Binary file packed with the scan angle and the extra precision required for the for a unique time stamp.

- Bare Earth Point Data

Data delivered in ASCII, fixed length, formatted files with one record per return as defined in Table 4 in the attached report. The records are ordered sequentially according to Easting with no duplicate records. The individual returns are classified according to the categories defined. This product was generated using an in house custom utility. The process involved extracting a Terrascan Binary file packed with the scan angle and the extra precision required for the for a unique time stamp. Bare Earth and Full Feature Hill-Shade Image Data delivered in GeoTIFF format with a TFW file. The image resolution is 1m. This product was generated in ArcView.

Process Date/Time: 2005-01-28 00:00:00

Process Step 2

CC ID: 1137616
Description:

Contour Data

Data is delivered at 50cm nominal contour interval, 2.5m labeled index contours in AutoCAD format. This product was generated using a combination of TerraModeler and Microstation.

Data Field 1:

Time

GPS time Reported to nearest microsecond

Data Field 2:

x,y location

Geographic location of return

NAD83-92, to nearest 0.01m

Data Field 3:

Elevation

Elevation of return NAVD-88, to nearest 0.01m

Data Field 4:

Return classification

Return classification of this return

First or last return

Data Field 5:

Off Nadir Angle

Angle between nadir and transmitted pulse

Reported to nearest 0.01 degrees

Data Field 6:

Return Intensity

Intensity of return

Data Field 7:

Classification Code

Classification of Return

Please note that all products, other than the hillshade products were delivered in 2 kilometer tiles and 1 kilometer tiles.

Return Classification System

Code Description Notes

2 Ground or water -Bare Earth surface

5 Vegetation -Non-ground points, 1.51 m up to 60 m above coincident ground

Process Date/Time: 2005-01-28 00:00:00

Process Step 3

CC ID: 1137617
Description:

Problems, Resolutions and Conclusions

Boresight

Our laser encountered some technical difficulties during acquisition. The laser manufacturer, Riegl, was unable to pin point the problem on the bench due to the small magnitude of the error. Riegl anticipated this result due to the limits of testing within a lab setting. As a precaution to eliminate any possible source of error, the laser angle encoder unit and the bearings were replaced. As an additional precaution, an extra screw was added during the inspection to improve the mounting of encoder mechanism. The unit belt was eliminated as the source of the problem. As a result of this malfunction, the boresight values for each flight had to be thoroughly reviewed. This manual process took approximately three weeks to resolve for the appropriate boresight corrections. The majority of the flights required a single roll correction of 0.097 degrees while others required an adjustment of 0.068 or 0.142 degrees. Certain flights required two-roll corrections.

Unfortunately, this boresight problem caused by equipment malfunction was only discovered after initially delivering the project - this was principally due to a shortcoming in our QC methodology that did not reveal mechanical problems such as those encountered with our scanning laser. It should be noted that the laser deployed for this project had previously operated without error on approximately 200 missions. We therefore consider the problems encountered with the South Bay project to be an outlier. Nevertheless, our processing flow has now been modified to identify such outliers by performing more quality control checks on the data prior to entering production. These include: profiles, full feature hillshades, GPS checks and distance grids. The distance grids are quite beneficial as they visually display relative elevation error between flightlines. Please note that we attempted to perform distance grids on all the tiles, but were successful on only 127 of 145 tiles due to lack of overlap in open water or tiles containing a great deal of flightlines causing the application to crash. Close attention was paid to these tiles in particular during the second round of production as to ensure the quality was good.

Process Date/Time: 2005-01-28 00:00:00

Process Step 4

CC ID: 1137618
Description:

The distance images were captured as screengrabs from the application and are located on the accompanying disk. The late discovery of this problem proved to be special challenge, as the processing team had spent the majority of the initial filtering run extracting the buildings from the point cloud. To deliver this project in a rapid manner, we opted to maintain the building classifications as to minimize the amount of reprocessing. Therefore we could not reprocess the point cloud with the boresight compensation values. An additional problem ensued; two GPS week rollovers occurred during acquisition. Due to software limitations we are unable to store GPS week, only GPS time, therefore, there is a strong likelihood of having multiple identical GPS times within the dataset. This situation leads to the possibility of associating the LiDAR data to the wrong trajectory and applying an incorrect roll compensation factor based upon aircraft position and required correction. To overcome these obstacles we developed a new procedure using existing software and custom applications. Once applied, these corrections were verified with the aforementioned distance grids. This issue is being addressed in future projects by allowing the LiDAR operator to tag flight lines with an attribute during acquisition. As a result of applying the roll compensation the overall quality of the dataset increased significantly allowing the project to meet the set accuracy guidelines. Upon the discovery of this problem by the USGS a few selected distance grids were generated. These grids indicated an average error of 30 centimeters while a few areas indicated errors of 45-50 centimeters. Once the boresight corrections were applied, all but a few areas are now within the range of +/- 15 centimeters. A few areas remain as outliers in the 15 to 30 cm range. A benefit of the correction was the elimination of "false vegetation". The error was more evident in flat open areas of the mud flats. The error manifested itself to look like isolated areas of low scrub. Once corrected these were reduced significantly to meet specification or eliminated all together. Vegetation removal also proved to be tricky as quite often the bulrush directly adjoining dikes masked themselves as extensions to the dikes. Pickle weed and other forms of low vegetation shorter than 20 cm also tended blend very well into the mudflats. Particular attention had to be paid to these areas and the assistance of the USGS proved to be a valuable asset in discerning valid ground from vegetation.

Process Date/Time: 2005-01-28 00:00:00

Process Step 5

CC ID: 1137619
Description:

The NOAA Office for Coastal Management (OCM) received the files in laz format from USGS via an FTP online repository. The files contained lidar elevation and intensity measurements. The data were in State Plane California Zone 3 and Zone 4, NAVD88 (orthometric) heights in meters. The California Coastal Project was divided into two projects: State Plane Zone 3 and State Plane Zone 4 respectively. OCM performed the following processing for data storage and Digital Coast provisioning purposes:

1. The data were converted from state plane coordinates to geographic coordinates.

2. The data were converted from NAVD88 (orthometric) heights in meters to GRS80 (ellipsoid) heights in meters using Geoid 09.

3. The LAS Noise class was dropped and Class 4 (medium vegetation scrutinized). All Class 4 points are considered to include all vegetation and man made objects.

Process Date/Time: 2016-02-10 00:00:00

Catalog Details

Catalog Item ID: 49635
GUID: gov.noaa.nmfs.inport:49635
Metadata Record Created By: Anne Ball
Metadata Record Created: 2017-11-15 15:21+0000
Metadata Record Last Modified By: SysAdmin InPortAdmin
Metadata Record Last Modified: 2022-08-09 17:11+0000
Metadata Record Published: 2022-03-16
Owner Org: OCMP
Metadata Publication Status: Published Externally
Do Not Publish?: N
Metadata Last Review Date: 2022-03-16
Metadata Review Frequency: 1 Year
Metadata Next Review Date: 2023-03-16