Cool Data Stuff at USGS

Slide 1

Slide 1 text

U.S. Department of the Interior U.S. Geological Survey Cool data stuff at USGS Rich Signell USGS Woods Hole, MA IOOS DMAC Steering Team Meeting Washington DC, Feb 27-28, 2013

Slide 2

Slide 2 text

USGS Mission  To provide geologic, topographic, and hydrologic information that contributes to the wise management of the Nation's natural resources and that promotes the health, safety, and well- being of the people  Slogan: “Science for a Changing World”

Slide 3

Slide 3 text

USGS Mission Areas alligned in 2010 with science themes  Climate and Land Use Change  Water  Ecosystems  Energy and Minerals  Environmental Health  Natural Hazards  Core Science Systems

Slide 4

Slide 4 text

USGS is a small agency  USGS $1.1B (FY12)  NOAA $5.5B, NASA $18B, USGOV $3,670B  USGS is $0.03 on a $100 dollar bill Total: $3,670B nationalpriorities.org

Slide 5

Slide 5 text

R&D Budget by Agency Source: http://www.aaas.org/spp/rd/fy2013/total13c.pdf MIL: $79.1B NASA: $9.1B NSF: $5.5B NOAA: $0.6B USGS: $0.6B EPA: $0.6B -------------- Total: $144B

Slide 6

Slide 6 text

Mission Area: Climate and Land Use Change

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Katama Bay Animation

Slide 13

Slide 13 text

ESRI RESTful Image Web Service

Slide 14

Slide 14 text

Same services accessed by ESRI client

Slide 15

Slide 15 text

The National Elevation Dataset (NED)

Slide 16

Slide 16 text

1/9-arc-second = ~ 3 m 1/3-arc-second = ~ 10 m 1-arc-second = ~ 30 m

Slide 17

Slide 17 text

NED is updated every 2 months

Slide 18

Slide 18 text

NED 1/9 second, Falmouth, MA 81 Queen St, 4.1m above MSL

Slide 19

Slide 19 text

In Aug 2012, NED entered the ocean… Adding bathy to the NED is a collaboration between NOAA/NGDC, USGS/CMG and USACE

Slide 20

Slide 20 text

Mission Area: Water

Slide 21

Slide 21 text

Old School

Slide 22

Slide 22 text

New School

Slide 23

Slide 23 text

New School

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

CIDA GDP Climate Downscaling Tool Built on: THREDDS OPeNDAP WCS WMS CSW WPS

Slide 26

Slide 26 text

Python & Matlab interfaces to GDP

Slide 27

Slide 27 text

CIDA Code on GitHub

Slide 28

Slide 28 text

Mission Area: Ecosystem Science

Slide 29

Slide 29 text

Mission Area: Energy and Minerals

Slide 30

Slide 30 text

Mission Area: Environmental Health Programs

Slide 31

Slide 31 text

Mission Area: Natural Hazards

Slide 32

Slide 32 text

Did You Feel It? Crowdsourcing

Slide 33

Slide 33 text

Superstorm Sandy Erosion Forecast USGS Coastal and Marine Program, St. Petersburg

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

228 surge sensors deployed by water offices in each state, 8 realtime, 950 verified high water marks Led by Brian McCallum, USGS Georgia Water Science Center

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

Mission Area: Core Science Systems

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

Community for Data Integration (CDI) $500K annually for proposals

Slide 40

Slide 40 text

Scientific Analysis tools at USGS  Majority of scientists use ArcGIS as their primary scientific analysis and visualation tool  ArcGIS 10 uses Python as it’s scripting and interface language  Use of Python allows ArcGIS users to utilize 100s of community packages  In 2012, USGS bought an Enthought Python Distribution (EPD) site license to facilitate interoperability of science workflows

Slide 41

Slide 41 text

OPeNDAP UGRID → ArcGIS TIN New 10.1 Python Toolbox import netCDF4 import arcpy netCDF4.Dataset(URL) arcpy.LandXMLToTin_3d(.. arcpy.TinRaster_3d(…

Slide 42

Slide 42 text

Assessing a 30 year ocean hindcast in the Gulf of Maine  One 15 TB dataset from 4D ocean model served via the THREDDS data server at UMASS/SMAST (commissioned by Mass CZM for Mass Ocean Plan)  600 time series datasets from 3 different THREDDS catalogs (USGS, NOAA/NMFS, WHOI) in Woods Hole

Slide 43

Slide 43 text

IOOS Model Data Interoperability Design ROMS POM WW3 WRF ECOM NcML NcML NcML NcML NcML Common Data Model OPeNDAP+CF WCS NetCDF Subset THREDDS Data Server Standardized (CF) Virtual Datasets Nonstandard Model Output Files (distributed) Web Services Matlab Panoply IDV ERDDAP Clients NetCDF -Java Library or Service NcML, NetCDF-Java, Common Data Model and THREDDS Data Server built and supported by Unidata under NSF-support WMS ncISO ArcGIS NetCDF4 -Python FVCOM Python EDC ERDDAP NetCDF-Java SOS GeoPortal Server GeoNetwork GI-CAT Gridded data, time series, point, profile, trajectory,swath

Slide 44

Slide 44 text

GI-CAT catalog broker service

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

Kellndorfer and Walker, 2012 Woods Hole Research Center NASA SRTM USGS NED USGS NLDC + Forest Service Ground Truth

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Conclusions  USGS & DMAC are on same services bandwagon (OGC, OPeNDAP/CF + ESRI + Custom)  USGS has a lot of cool data and products  But USGS should not be an organizing principle for data – we want to facilitate cross- agency, cross- sector, and cross-discipline mashups  Keep USGS CDI, NOAA DMIT, IOOS DMAC, CMSP (now Ocean Planning), OGC, ESIP circles overlapped  Keep building out the services, service-consuming web apps, science work flows, standards-based tools in Matlab, Python, R, ArcGIS and listening to scientists & developers