Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Standardized Framework for Reproducible Model Skill Assessment

A Standardized Framework for Reproducible Model Skill Assessment

Presentation at the 14th International Conference on Estuarine and Coastal Modeling (ECM14), University of Rhode Island, USA


Rich Signell

June 14, 2016

More Decks by Rich Signell

Other Decks in Science


  1. A Standardized Framework for Reproducible Model Skill Assessment Rich Signell

    (USGS, Woods Hole, MA) Filipe Fernandes (SECOORA) Kyle Wilcox (Axiom Data Science, Wickford, RI) 14th Conference on Estuarine and Coastal Modeling
  2. US Integrated Ocean Observing System (IOOS® ) IOOS® Plan defines:

    • Global Component • Coastal Component  17 Federal Agencies  11 Regional Associations
  3. IOOS Model Data Interoperability Design ROMS ADCIRC HYCOM SELFE SLOSH

    NcML NcML NcML NcML Common Data Model OPeNDAP+CF +UGRID|SGRID WCS NetCDF Subset THREDDS Data Server (TDS) Standardized (CF-1.6, UGRID-1.0, SGRID-0.3) Virtual Datasets Nonstandard Model Output Data Files Web Services Matlab Panoply IDV Clients NetCDF -Java Library or Broker WMS ncISO ArcGIS NetCDF4 -Python FVCOM Python ERDDAP NetCDF-Java SOS Geoportal Server GeoNetwork GI-CAT Observed data (buoy, gauge, ADCP, glider) Godiva2 pycsw-CKAN NcML Grid Sgrid Ugrid TimeSeries Profile Trajectory TimeSeriesProfile Nonstandard Data Files Catalog Services CMG Portal Delft3D NcML sudo apt-get docker docker run –d axiom/docker-thredds
  4. Catalog Search 4

  5. NCTOOLBOX: test_cf_ugrid3.m

  6. Skidaway “modena” glider: temperature

  7. Interoperable Access in Python (Iris)

  8. None
  9. None
  10. Automated model comparison

  11. 2015 Boston Light Swim 2015 Aug 15, 7:00 am start

    8 mile swim No wet suit How cold will the water be?
  12. NECOFS Massbay Forecast

  13. Reproducible Jupyter Notebook Go to: https://github.com/ioos/notebook_demos/boston_light_swim

  14. None
  15. None
  16. None
  17. Final Result

  18. None
  19. None
  20. Forecasts Sent to Swimmers

  21. Run it yourself

  22. Summary • Standardized framework makes skill assessment easy and powerful

    • Services are now easy to install and maintain • Skill assessment notebooks are reproducible by others (for free) • More assessment leads to more appropriate use of modeling products • More assessment of models leads to better models