Andy Cox
February 28, 2013
290

Prediction is very difficult, especially about the future

Presented at 2013 IRE CAR conference in Louisville, Kentucky.

Andy Cox

February 28, 2013

Transcript

1. Prediction is very difficult,
Andy Cox
The Weather Company
February 28, 2013
@InfoVizard

2. Who am I?

3. ?

4. Prediction and forecasting
Probability, uncertainty and confidence
Predictive modeling
Communicating uncertainty
Weather forecasting models
Goals

5. Goals
Not going to teach you how to do linear
regression or interpret a p-value

6. What is predictive modeling?
Working definition:
Using data to build an approximation of some
aspect of the real world that we can use to
better understand it or predict how it will
behave to previously unseen input.

7. So what?
Why should I care?

8. Probability
Don't be scared!

9. Basic probability

10. (Sample) size matters

11. 10 percent ≠ impossible
90 percent ≠ certain

12. Uncertainty is inherent

13. Essentially, all models are
wrong, but some are useful.
George E.P. Box

14. Uncertainty is inherent...
But people often want forecaster to take a stand

15. Uncertainty is inherent...
What should be communicated in a forecast?

16. Source: Coursera
Linear Regression

17. Predictive modeling dangers
Overfitting and overconfidence

18. Occam's Razor

19. Predictive modeling dangers
Overcoming with human + model approach

20. DICast Digit Topps
Integrated
first guess
Human-in-
the-loop
Public-oriented
aggregation
TWC Forecast System

21. Predictive modeling dangers
Data dredging

22. Meaningful? Or data dredging?

23. Correlation vs. causation

24. Source: kenpom.com

25. Forecast vs. prediction
Let's roll the dice

26. Forecast vs. prediction
Result of a single event vs. probability
distribution over the long term

27. Communicating
predictions

28. Communicating
predictions
National Hurricane Center

29. Communicating
predictions
National Weather Service
Storm Prediction Center

30. Source: NWS Storm Prediction Center

31. REMEMBER...A TORNADO WATCH MEANS CONDITIONS ARE FAVORABLE FOR
TORNADOES AND SEVERE THUNDERSTORMS IN AND CLOSE TO THE WATCH
AREA. PERSONS IN THESE AREAS SHOULD BE ON THE LOOKOUT FOR
THREATENING WEATHER CONDITIONS AND LISTEN FOR LATER STATEMENTS
AND POSSIBLE WARNINGS.

32. PRECAUTIONARY/PREPAREDNESS ACTIONS...
THE SAFEST PLACE TO BE DURING A TORNADO IS UNDER A WORKBENCH OR
OTHER PIECE OF STURDY FURNITURE. SEEK SHELTER ON THE LOWEST
FLOOR OF THE BUILDING IN AN INTERIOR HALLWAY OR ROOM SUCH AS A
CLOSET. USE BLANKETS OR PILLOWS TO COVER YOUR BODY AND ALWAYS
STAY AWAY FROM WINDOWS.
IF IN MOBILE HOMES OR VEHICLES...EVACUATE THEM AND GET INSIDE A
SUBSTANTIAL SHELTER. IF NO SHELTER IS AVAILABLE...LIE FLAT IN
THE NEAREST DITCH OR OTHER LOW SPOT AND COVER YOUR HEAD WITH

33. AT 6:57 PM CDT...A LARGE TORNADO WAS MOVING ALONG INTERSTATE 44
WEST OF NEWCASTLE. ON ITS PRESENT PATH...THIS LARGE DAMAGING
TORNADO WILL ENTER SOUTHWEST SECTIONS OF THE OKLAHOMA CITY
METRO AREA BETWEEN 7:15 AND 7:30 PM. PERSONS IN MOORE AND
SOUTH OKLAHOMA CITY SHOULD TAKE IMMEDIATE TORNADO PRECAUTIONS!
THIS IS AN EXTREMELY DANGEROUS AND LIFE THREATENING SITUATION.
IF YOU ARE IN THE PATH OF THIS LARGE AND DESTRUCTIVE TORNADO...
TAKE COVER IMMEDIATELY.

34. Numerical Weather
Prediction

35. Source: National Oceanic and Atmospheric Administration

36. Source: NWS Aviation Weather Center

37. Source: National Center for Atmospheric Research

38. Source: National Center for Atmospheric Research

39. Ensemble forecast plume
Source: http://eyewall.met.psu.edu/rich/gefs/Plumes.html

40. http://imgs.xkcd.com/comics/extrapolating.png

41. Thank you
@InfoVizard