27377 Bathymetry Alaska Data Set Published / External 26238 AKRO Sustainable Fisheries (SF) Division Metadata Portfolio Project In Work 2017 A Raster having 20 m resolution with decimal values was assembled from 18.6 billion bathymetric soundings that were obtained from the National Center for Environmental Information (NCEI) https://www.ncei.noaa.gov. Bathymetric soundings extends from Kuril-Kamchatka Trench in the Bearing Sea along the Aleutian Trench to the Gulf of Alaska, and in the Arctic Ocean from Prince Patrick Island to the International Date Line. Bathymetric soundings were scrutinized for accuracy using statistical analysis and visual inspection with some imputation. Editing processes included: deleting erroneous and superseded values, digitizing missing values, and referencing all data sets to a common, modern datum. Cartographic products for the entire Alaska Exclusive Economic Zone (AEEZ) and surrounding waters Time Period of Content Date: Various dates Maintenance and Update Frequency: As needed Spatial Extent of Data: Alaska and surrounding waters Keywords (Place): - Alaska Exclusive Economic Zone (AEEZ) (Theme): - Bathymetry - Depth, - Shaded relief - Alaska - Arctic Ocean - Aleutian Islands Attributes Table Detail - The attribute tables associated with each data point has four fields: (1) OBJECTID (is a unique, system-defined, object identifier number for each row in the table) (2) Value (Value of ocean depth multiplied by 100) (3) Count (the number of cells in the raster dataset with the cell value in the VALUE column (4) Z (the approximate depth of the water column at that spatial location). (Use Z value for depth) Data Quality Data Source(s): 86,000 separate bathymetric surveys including trackline geophysics, hydrographic surveys, gridded multi-beam NET CDF, MB-58 multi-beam sonar data, and satellite altimetry. Completeness: This dataset is kept up to date with data from National Center for Environmental Information (NCEI) data services Direct Spatial Reference Method: Raster Raster Object Type: Grid Cell Row Count: 338352 Column Count: 316476 XY Coordinate system NAD_1983_Alaska_Albers Datum D_North_American_1983 Linear Unit Meter (1.000000) Vertical Coordinate System NAD_1983 Spatial Domain Bounding Coordinates Southern Region: Kuril-Kamchatka Trench to Gulf of Alaska West_Bounding_Coordinate: 150 East_Bounding_Coordinate: -150 North_Bounding_Coordinate: 63 South_Bounding_Coordinate: 40 Bounding Coordinates Northern Region: Prince Patrick Island to the International Date Line West_Bounding_Coordinate: 150 East_Bounding_Coordinate: 0 North_Bounding_Coordinate: 80 South_Bounding_Coordinate: 71 This bathymetry grid was created in several steps: 1) Data were downloaded from NCEI and other online sources and parsed using NCEI Auto Grid, WGET with Python 2.7, Python 3.4, PostgreSQL DBMS, Quantum GIS, and ArcGIS. NetCDF, Grid, ASC and other data was converted into XYZ format. The XYZ coordinate files were projected to Albers Equal Area Conic projection and imported into an ESRI file geodatabase (FGDB) or individual tracklines. 2) The KNN model was set for one-hundred and fifty nearest neighbors and to delete the top 3% of those “outliers”. Percentiles were applied to clean data on a sliding scale based on standard deviation (StDEV). If the StDEV was lower – as in mud-flats or other relatively flat terrain, then a higher amount of the data’s trackline was selected for removal. Where the StDEV was high in high-relief areas, a smaller amount of the data was removed. The percentile technique was most effective in removing cross-band concavity or convexity in the survey’s trackline. Z-Score was often used to guide the GIS analyst in looking for areas of high variability that may warrant data imputation. PostGRESQL database functions such as ST_DWithin was used to perform the data removal because of its highly optimized indexing system, which allows it to perform the deletes at a rate several orders of magnitude greater than ArcGIS. PostGRESQL was not used for the actual Geodatabase because of its ArcGIS to DBMS interface overhead. When PostGRESQL was using to process data, it was all exported to ArcGIS File geodatabase. 3) Feature classes were combined using the ArcGIS Terrain data type to create a surface. The terrain feature type is based on triangulated irregular network (TIN). Varying terrains have been created based on the spacing of the data points. This includes 5 meter, 25 meter, and 50 meter. The 40 meter grid export is based on the 25 meter terrain. 4) The terrain has built in tools for locating integrated outliers using natural-neighbors (NN), slope and absolutes. Several iterations of NN and slope were applied to find potential outliers. Actual outliers were selected manually from the algorithm, cataloged, archived and then deleted from the dataset. The Terrain is rebuilt after each set of deletions. The external bounds of the TIN were trimmed to reflect the actual spatial extent of the point data. TIN was converted into a grid (raster with 200 meter resolution). Theme Alaska Theme Analytical Purposes Only Theme Bathy Theme Bathymetry Theme Depth Theme Ocean Theme SE Alaska Theme Southeast Alaska Regional Office Juneau AK USA call or email or visit web Data Set GIS dataset Point, Terrain, Raster Quarterly Map (digital) depth for analytical purposes only. NOT FOR NAVIGATION Steve Lewis, Jarvis Shultz ArcGIS Windows Soundings, multibeam Author 2013 Person Lewis, Steve steve.lewis@noaa.gov P.O. Box 21668 Juneau AK 99801 USA 907-586-7858 907-586-7255 web Data Steward 2015-12-24 Person Lewis, Steve steve.lewis@noaa.gov P.O. Box 21668 Juneau AK 99801 USA 907-586-7858 907-586-7255 email Distributor 2014 Person Lewis, Steve steve.lewis@noaa.gov P.O. Box 21668 Juneau AK 99801 USA 907-586-7858 907-586-7255 email Metadata Contact 2015-12-24 Person Lewis, Steve steve.lewis@noaa.gov P.O. Box 21668 Juneau AK 99801 USA 907-586-7858 907-586-7255 email Point of Contact 2015-12-24 Person Lewis, Steve steve.lewis@noaa.gov P.O. Box 21668 Juneau AK 99801 USA 907-586-7858 907-586-7255 email Ground Condition -133 170 88 40 Alaska and surrounding waters Continuing 2013 Continual Unclassified NA NA Not for Navigation By Raster format download via REST Services. Not for navigation. Analysis only. By Raster format download ALaskaBathy_SE REST Service for Alaska Fisheries bathymetry ESRI REST ESRI REST Service 8.2 Chked MD https://alaskafisheries.noaa.gov/arcgis/rest/services ShoreZoneFlex REST Service 2015-12-24T00:00:00 ESRI REST ESRI REST Service 8.2 10.3.1 Uncompressed Chked MD https://alaskafisheries.noaa.gov/arcgis/rest/services/bathy_40m/MapServer ALaskaBathy REST Service for Alaska Fisheries bathymetry 56gb Chked MD https://alaskafisheries.noaa.gov/arcgis/rest/services/bathy_40m/MapServer ALaskaBathy REST Service for Alaska Fisheries bathymetry ESRI REST ESRI REST Service 8.2 Chked MD https://alaskafisheries.noaa.gov/arcgis/rest/services/bathy_40m/MapServer Online Resource REST Service 2014 Updates Lewis, Steve continual updates 2013 Lewis, Steve Outlier removal processes In progress. Data was extracted, parsed, and groomed by each of the individual 84,009+ tracklines using statistical analysis and visual inspection with some imputation 86,600 MB tracklines were extracted and processed at full resolution, often less than 1 meter resolution We are currently downloading a multi-national bathymetric data with another 160,000 surveys getting ready to be added, Multibeam tracks are often prone to conal outliers. However, these outliers and investigated both using various GIS analytical tools 1/10 of a meter. Multibeam tracks are often prone with conal outliers. Used K-natural neighbors, Percentiles, and ArcGIS slope tools to location and remove outliers. Yes Unknown Yes No No 1 year NA Other NA NA -Currently, our process keeps a record of the survey from which that point originated. Some older data does not have this level of metadata. Servers are crawled for relevant data, and a list of download URLS to data files is returned. Data files are then retrieved using custom python scripts. Raw data is downloaded from online NCEI web servers. Data is converted to CSV or XYZ files Points missing one or more of their XYZ points values are removed and archived. Data is evaluated as a component of the bathymetry map to identify outliers (instances where data point(s) are not consistent with the expected variability of the surrounding environments) using a variety of statistical and manual methods. K-Natural Neighbors (KNN) - Python and SciPy Percentiles with Standard Deviations - Python and SciPy Slope and Neighbors - ArcGIS Models Manual/visual selection Human Imputation of points to make them consistent with surrounding terrain tracklines and satellite altimetry. Upon integration into the dataset, between 0-25% of the deepest and shallowest points are immediately removed from each trackline based on the StdDev of each induvial trackline. All data is archived; a dataset with the outliers could be built within 7 weeks. The percentage of points removed (R) is determined by a non-linear function of the datasets standard deviation (s), and can be seen below: R=-4.746813 +30.059/(1+(s/9.584625)^0.9983 ) This function was derived using a best-fit curve tool, which was instructed to return a naturally-logarithmic function which was equal to ~25 when s=0, and which decreased asymptotically to 0 as s grew larger. The function was then tailored to have what the developer felt was a reasonable slope The logic here is that datasets with low standard deviations would be relatively flat and featureless. Since they have a lower level of topographic complexity, they can undergo a higher rate of removal while still retaining the essential topographic character of the surface they represent. Data is then organized into a Kd-Tree structure in which data points are organized based on their values with the data sorted between levels in the tree (i.e., the first level is split along the x axis, the next level is split along the y axis, the next along the z axis, and then the fourth along the x axis again. The result was a tree which can be searched in O (log(n)) time, and which was optimized for quick spatial searches, critical for the next step. K-Nearest-Neighbors (KNN) statistical model is used on the data. This uses the value of each data point’s K spatially nearest neighbors (k-value) to produce an ‘expected value.’ The expected value is then subtracted from the point’s observed value, and the absolute value of this difference is the point’s ‘residual.’ After calculating the residuals for all of the points in the data set, we remove the 5% of points with the highest residuals. After these steps are completed, the remaining data are converted to Feature Classes. This data structure is composed of not only the raw data, but also a host of metadata calculated from the raw data, such as vessel name and tracklines number. Spatial indexes are added to the data to optimize operations. All internal data is point data and is stored in Alaska Albers project. ArcMap and Arc Pro Slope and Neighbor Outlier Tools. The ArcMap and ArcPro function analyzes each data point based on the slope of the rendered terrain polygon and the point’s immediate adjacent neighbors. If a sufficient portion of these slopes exceeded a manually pre-defined threshold, the point is flagged as a potential outlier but not removed. After this function has identified all potential outliers, the set is visually reviewed and flagged. Flagged points are manually removed from the terrain but stored as an independent shapefile and thus no data removed from the active dataset were truly deleted. 2015-09-22 Lewis, Steve can this dataset be used for navigation. No. gov.noaa.nmfs.inport:27377 Steve Lewis 2015-09-22T22:56:00 SysAdmin InPortAdmin 2025-05-15T19:15:56 2018-07-27 Alaska Regional Office AKRO 709 W. 9th St, P.O. Box 21668 Juneau AK 99802-1668 USA (907)586-7202 https://alaskafisheries.noaa.gov/ 8:00 am - 4:30 pm, Mon-Fri. 1001 Public No No 2018-07-27 1 Year 2019-07-27