A Standardized Framework for Reproducible Model Skill Assessment

Presentation at the 14th International Conference on Estuarine and Coastal Modeling (ECM14), University of Rhode Island, USA

Transcript

1. A Standardized Framework for Reproducible Model Skill Assessment Rich Signell (USGS, Woods Hole, MA) Filipe Fernandes (SECOORA) Kyle Wilcox (Axiom Data Science, Wickford, RI) 14th Conference on Estuarine and Coastal Modeling
2. US Integrated Ocean Observing System (IOOS® ) IOOS® Plan defines: • Global Component • Coastal Component  17 Federal Agencies  11 Regional Associations
3. IOOS Model Data Interoperability Design ROMS ADCIRC HYCOM SELFE SLOSH NcML NcML NcML NcML Common Data Model OPeNDAP+CF +UGRID|SGRID WCS NetCDF Subset THREDDS Data Server (TDS) Standardized (CF-1.6, UGRID-1.0, SGRID-0.3) Virtual Datasets Nonstandard Model Output Data Files Web Services Matlab Panoply IDV Clients NetCDF -Java Library or Broker WMS ncISO ArcGIS NetCDF4 -Python FVCOM Python ERDDAP NetCDF-Java SOS Geoportal Server GeoNetwork GI-CAT Observed data (buoy, gauge, ADCP, glider) Godiva2 pycsw-CKAN NcML Grid Sgrid Ugrid TimeSeries Profile Trajectory TimeSeriesProfile Nonstandard Data Files Catalog Services CMG Portal Delft3D NcML sudo apt-get docker docker run –d axiom/docker-thredds
4. Catalog Search 4
5. NCTOOLBOX: test_cf_ugrid3.m
6. Skidaway “modena” glider: temperature
7. Interoperable Access in Python (Iris)
10. Automated model comparison
11. 2015 Boston Light Swim 2015 Aug 15, 7:00 am start 8 mile swim No wet suit How cold will the water be?
12. NECOFS Massbay Forecast
13. Reproducible Jupyter Notebook Go to: https://github.com/ioos/notebook_demos/boston_light_swim
17. Final Result
20. Forecasts Sent to Swimmers
21. Run it yourself
22. Summary • Standardized framework makes skill assessment easy and powerful • Services are now easy to install and maintain • Skill assessment notebooks are reproducible by others (for free) • More assessment leads to more appropriate use of modeling products • More assessment of models leads to better models