2004 USGS Lidar: San Francisco Bay (CA)

Create custom data files by choosing data area, product type, map projection, file format, datum, etc.

Simple download of data files.

This graphic shows the lidar coverage for the 2004 USGS project along the southern area of San Francisco Bay.

Date that the source FGDC record was last modified.

Converted from FGDC Content Standards for Digital Geospatial Metadata (version FGDC-STD-001-1998) using 'fgdc_to_inport_xml.pl' script. Contact Tyler Christensen (NOS) for details.

Partial upload of Positional Accuracy fields only.

Partial upload to move data access links to Distribution Info.

The points are generated as Terrascan binary Format using Terrapoint's proprietary Laser Postprocessor Software. This software combines the Raw Laser file and GPS/IMU information to generate a point cloud for each individual flight. All the point cloud files encompassing the project area were then divided into 2 kilometer by 2-kilometer tiles. The referencing system of these tiles is based upon the project boundary minimum and maximums. This process is carried out in Terrascan.

The bald earth is subsequently extracted from the raw LiDAR points using Terrascan in a Microstation environment. The automated vegetation removal process takes place by building an iterative surface model. This surface model is generated using three main parameters: Building size, Iteration angle and Iteration distance. The initial model is based upon low points selected by a roaming window and are assumed to be ground points. The size of this roaming window is determined by the building size parameter. These low points are triangulated and the remaining points are evaluated and subsequently added to the model if they meet the Iteration angle and distance constraints (fig. 1). This process is repeated until no additional points are added within an iteration. There is also a maximum terrain angle constraint that determines the maximum terrain angle allowed within the model.

Quality Control

Once the data setup has taken place the manual quality control of the surface occurs. This process consists of visually examining the LiDAR points within Terrascan and correcting errors that occurred during the automated process. These corrections include verifying that all non ground elements, such as vegetation and buildings are removed from the ground model and that all small terrain undulations such as road beds, dykes, rock cuts and hill tops are present within the model. This process is done with the help of hillshades, contours, profiles and crosssections. To correct misclassifications, a full suite of Terrascan and custom in-house data tools are used.

Deliverables

All LiDAR Data Products were delivered on DVD-ROM. Three copies were provided. All products other than hill-shade data were provided in 1k and 2k tiles with a 25-meter buffer. Hill-shades were delivered in three large areas. Full Feature or All Return Point Data Data delivered in ASCII, comma delimited files with one record per return as defined in Table 4. The records are ordered sequentially according to Easting with no duplicate records. The individual returns are classified into the categories as defined in Table 5. If a return cannot be reliably classified as "Vegetation" or "Building/structure", it shall be assigned as "Not ground". This product was generated using an in house custom utility. The process involved extracting a Terrascan Binary file packed with the scan angle and the extra precision required for the for a unique time stamp.

- Bare Earth Point Data

Data delivered in ASCII, fixed length, formatted files with one record per return as defined in Table 4 in the attached report. The records are ordered sequentially according to Easting with no duplicate records. The individual returns are classified according to the categories defined. This product was generated using an in house custom utility. The process involved extracting a Terrascan Binary file packed with the scan angle and the extra precision required for the for a unique time stamp. Bare Earth and Full Feature Hill-Shade Image Data delivered in GeoTIFF format with a TFW file. The image resolution is 1m. This product was generated in ArcView.

Contour Data

Data is delivered at 50cm nominal contour interval, 2.5m labeled index contours in AutoCAD format. This product was generated using a combination of TerraModeler and Microstation.

Data Field 1:

Time

GPS time Reported to nearest microsecond

Data Field 2:

x,y location

Geographic location of return

NAD83-92, to nearest 0.01m

Data Field 3:

Elevation

Elevation of return NAVD-88, to nearest 0.01m

Data Field 4:

Return classification

Return classification of this return

First or last return

Data Field 5:

Off Nadir Angle

Angle between nadir and transmitted pulse

Reported to nearest 0.01 degrees

Data Field 6:

Return Intensity

Intensity of return

Data Field 7:

Classification Code

Classification of Return

Please note that all products, other than the hillshade products were delivered in 2 kilometer tiles and 1 kilometer tiles.

Return Classification System

Code Description Notes

2 Ground or water -Bare Earth surface

5 Vegetation -Non-ground points, 1.51 m up to 60 m above coincident ground

Problems, Resolutions and Conclusions

Boresight

Our laser encountered some technical difficulties during acquisition. The laser manufacturer, Riegl, was unable to pin point the problem on the bench due to the small magnitude of the error. Riegl anticipated this result due to the limits of testing within a lab setting. As a precaution to eliminate any possible source of error, the laser angle encoder unit and the bearings were replaced. As an additional precaution, an extra screw was added during the inspection to improve the mounting of encoder mechanism. The unit belt was eliminated as the source of the problem. As a result of this malfunction, the boresight values for each flight had to be thoroughly reviewed. This manual process took approximately three weeks to resolve for the appropriate boresight corrections. The majority of the flights required a single roll correction of 0.097 degrees while others required an adjustment of 0.068 or 0.142 degrees. Certain flights required two-roll corrections.

Unfortunately, this boresight problem caused by equipment malfunction was only discovered after initially delivering the project - this was principally due to a shortcoming in our QC methodology that did not reveal mechanical problems such as those encountered with our scanning laser. It should be noted that the laser deployed for this project had previously operated without error on approximately 200 missions. We therefore consider the problems encountered with the South Bay project to be an outlier. Nevertheless, our processing flow has now been modified to identify such outliers by performing more quality control checks on the data prior to entering production. These include: profiles, full feature hillshades, GPS checks and distance grids. The distance grids are quite beneficial as they visually display relative elevation error between flightlines. Please note that we attempted to perform distance grids on all the tiles, but were successful on only 127 of 145 tiles due to lack of overlap in open water or tiles containing a great deal of flightlines causing the application to crash. Close attention was paid to these tiles in particular during the second round of production as to ensure the quality was good.

The distance images were captured as screengrabs from the application and are located on the accompanying disk. The late discovery of this problem proved to be special challenge, as the processing team had spent the majority of the initial filtering run extracting the buildings from the point cloud. To deliver this project in a rapid manner, we opted to maintain the building classifications as to minimize the amount of reprocessing. Therefore we could not reprocess the point cloud with the boresight compensation values. An additional problem ensued; two GPS week rollovers occurred during acquisition. Due to software limitations we are unable to store GPS week, only GPS time, therefore, there is a strong likelihood of having multiple identical GPS times within the dataset. This situation leads to the possibility of associating the LiDAR data to the wrong trajectory and applying an incorrect roll compensation factor based upon aircraft position and required correction. To overcome these obstacles we developed a new procedure using existing software and custom applications. Once applied, these corrections were verified with the aforementioned distance grids. This issue is being addressed in future projects by allowing the LiDAR operator to tag flight lines with an attribute during acquisition. As a result of applying the roll compensation the overall quality of the dataset increased significantly allowing the project to meet the set accuracy guidelines. Upon the discovery of this problem by the USGS a few selected distance grids were generated. These grids indicated an average error of 30 centimeters while a few areas indicated errors of 45-50 centimeters. Once the boresight corrections were applied, all but a few areas are now within the range of +/- 15 centimeters. A few areas remain as outliers in the 15 to 30 cm range. A benefit of the correction was the elimination of "false vegetation". The error was more evident in flat open areas of the mud flats. The error manifested itself to look like isolated areas of low scrub. Once corrected these were reduced significantly to meet specification or eliminated all together. Vegetation removal also proved to be tricky as quite often the bulrush directly adjoining dikes masked themselves as extensions to the dikes. Pickle weed and other forms of low vegetation shorter than 20 cm also tended blend very well into the mudflats. Particular attention had to be paid to these areas and the assistance of the USGS proved to be a valuable asset in discerning valid ground from vegetation.

The NOAA Office for Coastal Management (OCM) received the files in laz format from USGS via an FTP online repository. The files contained lidar elevation and intensity measurements. The data were in State Plane California Zone 3 and Zone 4, NAVD88 (orthometric) heights in meters. The California Coastal Project was divided into two projects: State Plane Zone 3 and State Plane Zone 4 respectively. OCM performed the following processing for data storage and Digital Coast provisioning purposes:

1. The data were converted from state plane coordinates to geographic coordinates.

2. The data were converted from NAVD88 (orthometric) heights in meters to GRS80 (ellipsoid) heights in meters using Geoid 09.

3. The LAS Noise class was dropped and Class 4 (medium vegetation scrutinized). All Class 4 points are considered to include all vegetation and man made objects.