Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Prediction is very difficult, especially about the future

Andy Cox
February 28, 2013

Prediction is very difficult, especially about the future

Presented at 2013 IRE CAR conference in Louisville, Kentucky.

Andy Cox

February 28, 2013
Tweet

More Decks by Andy Cox

Other Decks in Science

Transcript

  1. Prediction is very difficult,
    especially about the future
    Andy Cox
    The Weather Company
    February 28, 2013
    @InfoVizard

    View Slide

  2. Who am I?

    View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. ?

    View Slide

  13. View Slide

  14. Prediction and forecasting
    Probability, uncertainty and confidence
    Predictive modeling
    Communicating uncertainty
    Weather forecasting models
    Goals

    View Slide

  15. Goals
    Not going to teach you how to do linear
    regression or interpret a p-value

    View Slide

  16. View Slide

  17. What is predictive modeling?
    Working definition:
    Using data to build an approximation of some
    aspect of the real world that we can use to
    better understand it or predict how it will
    behave to previously unseen input.

    View Slide

  18. So what?
    Why should I care?

    View Slide

  19. Probability
    Don't be scared!

    View Slide

  20. Basic probability

    View Slide

  21. View Slide

  22. View Slide

  23. View Slide

  24. View Slide

  25. View Slide

  26. View Slide

  27. (Sample) size matters

    View Slide

  28. View Slide

  29. 10 percent ≠ impossible
    90 percent ≠ certain

    View Slide

  30. Uncertainty is inherent
    Ignore at your own peril

    View Slide

  31. Essentially, all models are
    wrong, but some are useful.
    George E.P. Box

    View Slide

  32. View Slide

  33. Uncertainty is inherent...
    But people often want forecaster to take a stand

    View Slide

  34. View Slide

  35. Uncertainty is inherent...
    What should be communicated in a forecast?

    View Slide

  36. Source: Coursera
    Linear Regression

    View Slide

  37. Predictive modeling dangers
    Overfitting and overconfidence

    View Slide

  38. View Slide

  39. Occam's Razor

    View Slide

  40. Predictive modeling dangers
    Overcoming with human + model approach

    View Slide

  41. DICast Digit Topps
    Integrated
    first guess
    Human-in-
    the-loop
    Public-oriented
    aggregation
    TWC Forecast System

    View Slide

  42. Predictive modeling dangers
    Data dredging

    View Slide

  43. Meaningful? Or data dredging?

    View Slide

  44. Correlation vs. causation

    View Slide

  45. View Slide

  46. View Slide

  47. View Slide

  48. Source: kenpom.com

    View Slide

  49. Forecast vs. prediction
    Let's roll the dice

    View Slide

  50. Forecast vs. prediction
    Result of a single event vs. probability
    distribution over the long term

    View Slide

  51. View Slide

  52. Communicating
    predictions

    View Slide

  53. Communicating
    predictions
    National Hurricane Center

    View Slide

  54. View Slide

  55. View Slide

  56. View Slide

  57. Communicating
    predictions
    National Weather Service
    Storm Prediction Center

    View Slide

  58. View Slide

  59. View Slide

  60. View Slide

  61. View Slide

  62. Source: NWS Storm Prediction Center

    View Slide

  63. View Slide

  64. REMEMBER...A TORNADO WATCH MEANS CONDITIONS ARE FAVORABLE FOR
    TORNADOES AND SEVERE THUNDERSTORMS IN AND CLOSE TO THE WATCH
    AREA. PERSONS IN THESE AREAS SHOULD BE ON THE LOOKOUT FOR
    THREATENING WEATHER CONDITIONS AND LISTEN FOR LATER STATEMENTS
    AND POSSIBLE WARNINGS.
    Tornado watch

    View Slide

  65. View Slide

  66. PRECAUTIONARY/PREPAREDNESS ACTIONS...
    THE SAFEST PLACE TO BE DURING A TORNADO IS UNDER A WORKBENCH OR
    OTHER PIECE OF STURDY FURNITURE. SEEK SHELTER ON THE LOWEST
    FLOOR OF THE BUILDING IN AN INTERIOR HALLWAY OR ROOM SUCH AS A
    CLOSET. USE BLANKETS OR PILLOWS TO COVER YOUR BODY AND ALWAYS
    STAY AWAY FROM WINDOWS.
    IF IN MOBILE HOMES OR VEHICLES...EVACUATE THEM AND GET INSIDE A
    SUBSTANTIAL SHELTER. IF NO SHELTER IS AVAILABLE...LIE FLAT IN
    THE NEAREST DITCH OR OTHER LOW SPOT AND COVER YOUR HEAD WITH
    YOUR HANDS.
    Tornado warning

    View Slide

  67. AT 6:57 PM CDT...A LARGE TORNADO WAS MOVING ALONG INTERSTATE 44
    WEST OF NEWCASTLE. ON ITS PRESENT PATH...THIS LARGE DAMAGING
    TORNADO WILL ENTER SOUTHWEST SECTIONS OF THE OKLAHOMA CITY
    METRO AREA BETWEEN 7:15 AND 7:30 PM. PERSONS IN MOORE AND
    SOUTH OKLAHOMA CITY SHOULD TAKE IMMEDIATE TORNADO PRECAUTIONS!
    THIS IS AN EXTREMELY DANGEROUS AND LIFE THREATENING SITUATION.
    IF YOU ARE IN THE PATH OF THIS LARGE AND DESTRUCTIVE TORNADO...
    TAKE COVER IMMEDIATELY.
    Tornado warning

    View Slide

  68. Numerical Weather
    Prediction

    View Slide

  69. Source: National Oceanic and Atmospheric Administration

    View Slide

  70. Source: NWS Aviation Weather Center

    View Slide

  71. Source: National Center for Atmospheric Research

    View Slide

  72. Source: National Center for Atmospheric Research

    View Slide

  73. View Slide

  74. View Slide

  75. Ensemble forecast plume
    Source: http://eyewall.met.psu.edu/rich/gefs/Plumes.html

    View Slide

  76. View Slide

  77. http://imgs.xkcd.com/comics/extrapolating.png

    View Slide

  78. Thank you
    @InfoVizard

    View Slide