Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Prediction is very difficult, especially about the future

4f36a55babc014c17fb69cca9bf4a9c2?s=47 Andy Cox
February 28, 2013

Prediction is very difficult, especially about the future

Presented at 2013 IRE CAR conference in Louisville, Kentucky.

4f36a55babc014c17fb69cca9bf4a9c2?s=128

Andy Cox

February 28, 2013
Tweet

Transcript

  1. Prediction is very difficult, especially about the future Andy Cox

    The Weather Company February 28, 2013 @InfoVizard
  2. Who am I?

  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. None
  10. None
  11. None
  12. ?

  13. None
  14. Prediction and forecasting Probability, uncertainty and confidence Predictive modeling Communicating

    uncertainty Weather forecasting models Goals
  15. Goals Not going to teach you how to do linear

    regression or interpret a p-value
  16. None
  17. What is predictive modeling? Working definition: Using data to build

    an approximation of some aspect of the real world that we can use to better understand it or predict how it will behave to previously unseen input.
  18. So what? Why should I care?

  19. Probability Don't be scared!

  20. Basic probability

  21. None
  22. None
  23. None
  24. None
  25. None
  26. None
  27. (Sample) size matters

  28. None
  29. 10 percent ≠ impossible 90 percent ≠ certain

  30. Uncertainty is inherent Ignore at your own peril

  31. Essentially, all models are wrong, but some are useful. George

    E.P. Box
  32. None
  33. Uncertainty is inherent... But people often want forecaster to take

    a stand
  34. None
  35. Uncertainty is inherent... What should be communicated in a forecast?

  36. Source: Coursera Linear Regression

  37. Predictive modeling dangers Overfitting and overconfidence

  38. None
  39. Occam's Razor

  40. Predictive modeling dangers Overcoming with human + model approach

  41. DICast Digit Topps Integrated first guess Human-in- the-loop Public-oriented aggregation

    TWC Forecast System
  42. Predictive modeling dangers Data dredging

  43. Meaningful? Or data dredging?

  44. Correlation vs. causation

  45. None
  46. None
  47. None
  48. Source: kenpom.com

  49. Forecast vs. prediction Let's roll the dice

  50. Forecast vs. prediction Result of a single event vs. probability

    distribution over the long term
  51. None
  52. Communicating predictions

  53. Communicating predictions National Hurricane Center

  54. None
  55. None
  56. None
  57. Communicating predictions National Weather Service Storm Prediction Center

  58. None
  59. None
  60. None
  61. None
  62. Source: NWS Storm Prediction Center

  63. None
  64. REMEMBER...A TORNADO WATCH MEANS CONDITIONS ARE FAVORABLE FOR TORNADOES AND

    SEVERE THUNDERSTORMS IN AND CLOSE TO THE WATCH AREA. PERSONS IN THESE AREAS SHOULD BE ON THE LOOKOUT FOR THREATENING WEATHER CONDITIONS AND LISTEN FOR LATER STATEMENTS AND POSSIBLE WARNINGS. Tornado watch
  65. None
  66. PRECAUTIONARY/PREPAREDNESS ACTIONS... THE SAFEST PLACE TO BE DURING A TORNADO

    IS UNDER A WORKBENCH OR OTHER PIECE OF STURDY FURNITURE. SEEK SHELTER ON THE LOWEST FLOOR OF THE BUILDING IN AN INTERIOR HALLWAY OR ROOM SUCH AS A CLOSET. USE BLANKETS OR PILLOWS TO COVER YOUR BODY AND ALWAYS STAY AWAY FROM WINDOWS. IF IN MOBILE HOMES OR VEHICLES...EVACUATE THEM AND GET INSIDE A SUBSTANTIAL SHELTER. IF NO SHELTER IS AVAILABLE...LIE FLAT IN THE NEAREST DITCH OR OTHER LOW SPOT AND COVER YOUR HEAD WITH YOUR HANDS. Tornado warning
  67. AT 6:57 PM CDT...A LARGE TORNADO WAS MOVING ALONG INTERSTATE

    44 WEST OF NEWCASTLE. ON ITS PRESENT PATH...THIS LARGE DAMAGING TORNADO WILL ENTER SOUTHWEST SECTIONS OF THE OKLAHOMA CITY METRO AREA BETWEEN 7:15 AND 7:30 PM. PERSONS IN MOORE AND SOUTH OKLAHOMA CITY SHOULD TAKE IMMEDIATE TORNADO PRECAUTIONS! THIS IS AN EXTREMELY DANGEROUS AND LIFE THREATENING SITUATION. IF YOU ARE IN THE PATH OF THIS LARGE AND DESTRUCTIVE TORNADO... TAKE COVER IMMEDIATELY. Tornado warning
  68. Numerical Weather Prediction

  69. Source: National Oceanic and Atmospheric Administration

  70. Source: NWS Aviation Weather Center

  71. Source: National Center for Atmospheric Research

  72. Source: National Center for Atmospheric Research

  73. None
  74. None
  75. Ensemble forecast plume Source: http://eyewall.met.psu.edu/rich/gefs/Plumes.html

  76. None
  77. http://imgs.xkcd.com/comics/extrapolating.png

  78. Thank you @InfoVizard