A Distributed, Standards-Based Framework for Searching, Accessing, Analyzing and Visualizing Met-Ocean Data: Application to Hurricane Sandy Rich Signell and Andrew Yan, USGS, USA Filipe Fernandes, Centro Universidade Monte Serrat, Brazil. Kyle Wilcox, Axiom Data Science, USA 

US Integrated Ocean Observing System (IOOS® )  17 Federal Agencies  11 Regional Associations 

From pile of files to interoperability Search and Browse Datasets Access Datasets Dataset files 

Ocean grids are often not regularly spaced! Stretched surface and terrain following vertical coordinates Curvilinear orthogonal horizontal coordinates 

CF Conventions for met/ocean models Groups using CF: OGC: Open Geospatial Consortium GO-ESSP: Global Organization for Earth System Science Portal IOOS: Integrated Ocean Observing System float temp(ocean_time, s_rho, eta_rho, xi_rho) ; temp: standard_name = “sea_water_potential_temperature" ; temp: units = "Celsius" ; temp: coordinates = "lon_rho lat_rho s_rho ocean_time" ; double s_rho(s_rho) ; s_rho: long_name = "S-coordinate at RHO-points" ; s_rho: positive = "up" ; s_rho: standard_name = "ocean_s_coordinate_g1" ; s_rho: formula_terms = "s: s_rho C: Cs_r eta: zeta depth: h depth_c: hc " 

IOOS Recommended Web Services and Data Encodings In-situ data (buoys, piers, towed sensors) Gridded data (model outputs, satellite) OGC Sensor Observation Service (SOS) OPeNDAP with Climate and Forecast Conventions XML or CSV Binary DAP using Climate and Forecast (CF) conventions Images of data OGC Web Map Service (WMS) GeoTIFF, PNG etc. -possibly with standardized styles Data Type Web Service Encoding 

IOOS Data Infrastructure Diagram ROMS ADCIRC HYCOM SELFE NCOM NcML NcML NcML NcML NcML Common Data Model OPeNDAP+CF WCS NetCDF Subset THREDDS Data Server Standardized (CF-1.6, UGRID-0.9) Virtual Datasets Nonstandard Model Output Data Files Web Services Matlab Panoply IDV Clients NetCDF -Java Library or Broker WMS ncISO ArcGIS NetCDF4 -Python FVCOM Python ERDDAP NetCDF-Java SOS Geoportal Server GeoNetwork GI-CAT Observed data (buoy, gauge, ADCP, glider) Godiva2 CKAN-pyCSW NcML Grid Ugrid TimeSeries Profile Trajectory TimeSeriesProfile Nonstandard Data Files Catalog Services 

WMS-driven Model Viewing Portal 

Iris: Data Access using OPeNDAP+CF: 

Interoperable Access “viridi”!!! 

Two things that CF doesn’t do • Unstructured Grids • Staggered Grids 

Standardized access to Unstructured Grid data 

SGRID Conventions: github/sgrid 

sci-wms (1.0 release) 

UGRID browsing using sci-wms 

SGRID browsing using sci-wms 

UGRID data access using pyugrid 

UGRID access using pyugrid 

SGRID access using pysgrid 

SGRID data access with pysgrid 

pycsw for search 

OWSlib CSW 

Browse with Github.com/TerriaJS Consumes: WMS, ESRI Rest, OSM, KML, GeoJSON, GPX, even CSV files Can search CKAN to build dynamic catalog Supports both 3D and 2D in browser 

IOOS binstar channel: 139 packages 

yaml2ncml.py 

Summary • We have now a complete open source framework for model data delivery, search, browse and access • Need to make it easy for modelers to connect their data • Standardized services and tools benefit both scientists and developers • Join us for sprint tomorrow on UGRID! 

Search “GitHub” “Signell” 

Client Software Stack • Environment: – IPython Notebooks, Anaconda, Binstar, Github • Delivery – THREDDS Data Server, – OPeNDAP +CF +UGRID +SGRID • Access – Iris, pyugrid, pysgrid • Search – CSW using pycsw (server) & OWSLib (client) • Browse – Sciwms (server) & terriaJS (client) 

Interoperable Model Comparison in Matlab (using nctoolbox)