Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scientific Computing in [Operational] Meteorology

Scientific Computing in [Operational] Meteorology

A combination of things I've learned over the years. Content comes from the paper, "Best Practices in Scientific Computing" (http://arxiv.org/abs/1210.0530), Software Carpentry (https://software-carpentry.org), and my own trial-and-error experiences.

Patrick Marsh

April 13, 2015
Tweet

Transcript

  1. Things  I  Wish  I  Had  Known   10  Years  Ago

    Patrick  Marsh  (@pmarshwx) NOAA/NWS  Storm  Prediction  Center
  2. A  Little  About  Me • BS  in  Physics  &  Mathematics

     from  University  of  Arkansas • MS/PhD  in  Meteorology  from  University  of  Oklahoma • NSSL  Liaison  to  the  Hazardous  Weather  Testbed (2009-­‐2013) • VORTEX2  Operations  Center  (2009-­‐2010) • NWS  Storm  Prediction  Center  (2013-­‐????)
  3. A  Little  About  Me • BS  in  Physics  &  Mathematics

     from  University  of  Arkansas • MS/PhD  in  Meteorology  from  University  of  Oklahoma • NSSL  Liaison  to  the  Hazardous  Weather  Testbed (2009-­‐2013) • VORTEX2  Operations  Center  (2009-­‐2010) • NWS  Storm  Prediction  Center  (2013-­‐????) Everything  in  yellow  required/requires  significant  programming  
  4. “Always  code  as  if  the  person  who  ends   up

     maintaining  your  code  is  a  violent   psychopath  who  knows  where  you  live.”
  5. Motivation • Software  is  lab  equipment  for  the  21st  Century

    • Scientists  spend  a  lot  of  time  writing  it • But  over  90%  are  self-­‐taught • We  don't  know  what  good looks  like
  6. What  follows  is  derived  from  the  paper, Best  Practices  for

     Scientific  Computing http://arxiv.org/abs/1210.0530 and Software  Carpentry https://software-­‐carpentry.org and Some  Meteorology-­‐specific  advice   learned  through  trial  and  error
  7. Write  Programs  for  People, not  Computers Code  that  is  hard

     to  understand   makes  it  hard  to  know  if  it  is  working  correctly.
  8. Write  Programs  for  People, not  Computers Code  that  is  hard

     to  understand is  hard  to  use,  even  for  your  future  self.
  9. Write  Programs  for  People, not  Computers A  program  should  not

     require  its  readers  to  hold  more  than  a   handful  of  facts  in  memory  at  once.
  10. Don’t  Repeat  Yourself (or  others) Anything  repeated  in  two  or

     more  places  will  eventually  be   wrong  in  at  least  one.
  11. Don’t  Repeat  Yourself (or  others) Every  piece  of  data  must

     have  a  single,  authoritative   representation.
  12. Don’t  Repeat  Yourself (or  others) Look  for  methods/solutions  outside  of

     Meteorology. Model  Grids  ==  Images Model  Grid  Cell  ==  Image  Pixel
  13. Optimize  Software   Only  After  it  Works  Correctly Even  experts

     find  it  hard  to  predict  performance  bottlenecks
  14. Optimize  Software   Only  After  it  Works  Correctly Write  code

     in  the  highest-­‐level  language  possible.
  15. Learn  a  Multitude  of (Meteorological)  Data  Formats   Science  is

     becoming  more  collaborative.  Inevitably,  your   collaborator  will  not  use  the  same  data  format  you  use.
  16. Learn  a  Multitude  of (Meteorological)  Data  Formats   Research  models

     (typically)  output  netcdf;   NWS  AWIPS  uses  grib2;  NWS  AWIPS  2  uses  HDF5; NWS  SPC  uses  GEMPAK; Harold  Brooks  uses  Microsoft  Excel.
  17. It  Will  Always  Take  Longer  Than  You  Think Hofstadter's  Law:

     It  always  takes  longer  than  you  expect,  even   when  you  take  into  account  Hofstadter's  Law.