Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A free system for visualization and analysis of distributed model output in your browser

Rich Signell
December 16, 2015

A free system for visualization and analysis of distributed model output in your browser

Presentation to NOAA Modeling Groups as part of the "Tuesday" lunch seminar series

Rich Signell

December 16, 2015
Tweet

More Decks by Rich Signell

Other Decks in Science

Transcript

  1. A free system for visualization and analysis of distributed model

    output in your browser Rich Signell , USGS, Woods Hole, MA, USA Filipe Fernandes, Centro Universidade Monte Serrat, Santos, Brazil Kyle Wilcox, Axiom Data Science, Wickford, RI, USA NOAA “Tuesday” Seminar Series 2015-12-16
  2. The Fourth Paradigm: eScience 1. Thousand years ago: science was

    empirical describing natural phenomena 2. Last few hundred years: + theoretical branch using models, generalizations 3. Last few decades: + computational branch simulating complex phenomena 4. Today: + data exploration branch (eScience) • Data captured by instruments or simulations • Processed by software • Information/knowledge stored in computer • Scientist analyzes database / files using data management and statistics Ref: Slide from Turing Award Winner Jim Gray’s presentation to the NRC, Jan 11, 2007 (last presentation - lost at sea Jan 28)
  3. The 4th Network Layer: Data • “We need an end-to-end,

    layer-by-layer, designed information technology … that are composed of no more than a stack of protocols” • “We need open standards… and above all, we need to teach scientists to work in this new layer of data” 3 From the essay: “I have seen the Paradigm Shift, and It Is Us”, byJohn Wilbanks, in the book “The Fourth Paradigm” Data Web TCP/IP Ethernet
  4. US Integrated Ocean Observing System (IOOS® ) IOOS® Plan defines:

    • Global Component • Coastal Component  17 Federal Agencies  11 Regional Associations
  5. IOOS Core Principles • Adopt open standards & practices •

    Avoid customer-specific stovepipes • Standardized access services implemented at data providers 5 Customer Web access service Data Provider Observations Models
  6. Ocean grids are often not regularly spaced! Stretched surface and

    terrain following vertical coordinates Curvilinear orthogonal horizontal coordinates
  7. NetCDF Climate and Forecast (CF) Conventions + UGRID + SGRID

    Groups using CF: GO-ESSP: Global Organization for Earth System Science Portal IOOS: Integrated Ocean Observing System ESMF: Earth System Modeling Framework OGC: Open Geospatial Consortium (GALEON: WCS profile)
  8. Time Series, Trajectories Meteorology and Wave Buoy in the Gulf

    of Maine. Image courtesy of NOAA. Ocean Glider. Photo by Dave Fratantoni, Woods Hole Oceanographic Institution
  9. OGC Sensor Observation Service (SOS) • Provides standard access to

    sensor data – GetCapabilities: provides the means to access SOS service metadata – DescribeSensor - retrieves detailed information about the sensors and processes generating those measurements. – GetObservation - provides access to sensor observations and measurement data via a spatio- temporal query that can be filtered by phenomena
  10. IOOS Recommended Web Services and Data Encodings In-situ data (buoys,

    piers, towed sensors) Gridded data (model outputs, satellite) OGC Sensor Observation Service (SOS) OPeNDAP with Climate and Forecast Conventions XML or CSV Binary DAP using Climate and Forecast (CF) conventions Images of data OGC Web Map Service (WMS) GeoTIFF, PNG etc. -possibly with standardized styles Data Type Web Service Encoding
  11. IOOS Data Infrastructure Diagram ROMS ADCIRC HYCOM SELFE NCOM NcML

    NcML NcML NcML NcML Common Data Model OPeNDAP+CF WCS NetCDF Subset THREDDS Data Server Standardized (CF-1.6, UGRID-0.9) Virtual Datasets Nonstandard Model Output Data Files Web Services Matlab Panoply IDV Clients NetCDF -Java Library or Broker WMS ncISO ArcGIS NetCDF4 -Python FVCOM Python ERDDAP NetCDF-Java SOS Geoportal Server GeoNetwork GI-CAT Observed data (buoy, gauge, ADCP, glider) Web Portals CKAN-pyCSW NcML Grid Ugrid TimeSeries Profile Trajectory TimeSeriesProfile Nonstandard Data Files Catalog Services
  12. 2015 Boston Light Swim, Aug 15, 7:00am since 1907, 8

    miles, no wet suit How cold will the water be?
  13. Benefits of Standards-Based, Catalog-Driven, Reproducible Workflows • Find the real

    problems – Easy problems that can be fixed in minutes to day – Harder problems to guide future work • Fixes for specific workflows benefit everyone • Build success stories • Create reproducible workflows that others can learn from, expand on, or transform • Standardized workflows help develop the 4th network layer for data