NATO CMRE Presentation

Presentation on Oct 29, 2014 at NATO CMRE

Transcript

1. Ocean, Atmosphere and Climate Model Assessment for Everyone: A distributed, standards-based framework for finding, accessing and using modeled and observed data Rich Signell (USGS), Woods Hole, MA NATO PTO-CMRE: October 29, 2014
2. US Integrated Ocean Observing System (IOOS® ) IOOS® Plan defines: • Global Component • Coastal Component  17 Federal Agencies  11 Regional Associations
3. IOOS Core Principles • Adopt open standards & practices • Avoid customer-specific stovepipes • Standardized access services implemented at data providers 3 Customer Web access service Data Provider Observations Models
4. Issue: Ocean grids are not regularly spaced! Stretched surface and terrain following vertical coordinates Curvilinear orthogonal horizontal coordinates
5. Time Series, Trajectories Meteorology and Wave Buoy in the Gulf of Maine. Image courtesy of NOAA. Ocean Glider. Photo by Dave Fratantoni, Woods Hole Oceanographic Institution
6. NetCDF Climate and Forecast (CF) Conventions provide a solution Groups using CF: GO-ESSP: Global Organization for Earth System Science Portal IOOS: Integrated Ocean Observing System ESMF: Earth System Modeling Framework OGC: Open Geospatial Consortium (GALEON: WCS profile)
7. NcML to the Rescue (XML markup language for NetCDF) <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"> <aggregation dimName="time" type="joinExisting"> <netcdf location="http://rocky.umeoce.maine.edu/cdfs/gomoos.20050509.cdf" /> <netcdf location="http://rocky.umeoce.maine.edu/cdfs/gomoos.20050510.cdf" /> </aggregation> <variable name="elev"> <attribute name="coordinates" value="x y" /> </variable> <variable name="depth"> <attribute name="coordinates" value="x y" /> </variable> <variable name="temp"> <attribute name="coordinates" value="x y zpos" /> </variable> <variable name="time"> <attribute name="units" value="days since 2002-01-01 00:00 UTC" /> </variable> <variable name="zpos"> <attribute name="standard_name" value="ocean_sigma_coordinate" /> <attribute name="formula_terms" value="sigma: zpos eta: elev depth: depth" /> <attribute name="positive" value="up" /> <attribute name="units" value="1" /> <attribute name="axis" value="Z" /> <values> 0 -0.002 -0.004 -0.008 -0.016 -0.024 -0.032 -0.04 -0.06 -0.08 -0.10 -0.12 -0.14 -0.16 </variable>
8. IOOS Recommended Web Services and Data Encodings In-situ data (buoys, piers, towed sensors) Gridded data (model outputs, satellite) OGC Sensor Observation Service (SOS) OPeNDAP with Climate and Forecast Conventions XML or CSV Binary DAP using Climate and Forecast (CF) conventions Images of data OGC Web Map Service (WMS) GeoTIFF, PNG etc. -possibly with standardized styles Data Type Web Service Encoding
9. IOOS Model Data Interoperability Design ROMS ADCIRC HYCOM SELFE NCOM NcML NcML NcML NcML NcML Common Data Model OPeNDAP+CF WCS NetCDF Subset THREDDS Data Server Standardized (CF-1.6, UGRID-0.9) Virtual Datasets Nonstandard Model Output Data Files Web Services Matlab Panoply IDV Clients NetCDF -Java Library or Broker WMS ncISO ArcGIS NetCDF4 -Python FVCOM Python ERDDAP NetCDF-Java SOS Geoportal Server GeoNetwork GI-CAT Observed data (buoy, gauge, ADCP, glider) Godiva2 CKAN NcML Grid Ugrid TimeSeries Profile Trajectory TimeSeriesProfile Nonstandard Data Files Catalog Services
10. WMS Browsing with THREDDS/ncWMS
11. Matlab NCTOOLBOX https://github.com/nctoolbox • Objective: Make it simple to access CF data • Example function: • [t, geo]=nj_tslice(URL,’temp’,1); • t = 22x120x180 single • geo = – lat: [120x180 single] – lon: [120x180 single] – z: [22x120x180 double] – time: 733582 (matlab datenum) • nj_tslice works identically for ROMS, HYCOM, Delft3D, POM, NCOM, WRF, Wavewatch3 • URL can be: local NetCDF, remote NetCDF, NcML, OpenDAP Data URL
12. Comparing Models with Data in Matlab Model 1: UMASS-ECOM Model 2: UMAINE-POM Data: SST 2008-Sep-08 07:32
13. Skidaway modena glider (Sep 11-23)
14. compare_secoora_model_sections.m (using nc_genslice.m)
15. 3D visualization of data with IDV
16. Iris Python tools from the UK Met Office
17. ESPreSSO model (Rutgers)
18. Global RTOFS (NOAA)
19. IOOS Models Notebook on Wakari
20. GI-CAT catalog broker service architecture Searching for Data
24. Automated model comparison
25. ERDDAP Browse
26. ERDDAP Data Extraction
27. ERDDAP to Ocean Data View
28. CMRE Catalog Search
29. CMRE Catalog Search
30. Key Infrastructure Components • Common data models for “feature types” (structured and unstructured grids, time series, profiles, swaths) (Unidata CDM) • Standard web data services for delivering these common data model “feature types” • Standard catalog services for the metadata (OGC, Unidata) • Tools to search, access and process these services in common analysis environments: R, Matlab, Python, ArcGIS, JavaScript
31. Infrastructure Benefits • What are the benefits? – Lots of choices for data access (Browser, Matlab, Python, Excel, IDV, R, IDL) – Less time wasted messing with data, more time spent on science – More usage of model results by more people – Faster feedback to modelers, leading to improved models – Better science, better models, better world
32. It’s all on GitHub + rsignell