Optimal Forecast Reconciliation

Talk given at UNSW, 25 August 2017

Transcript

1. Op mal Forecast Reconcilia on Rob J Hyndman UNSW: 25 August 2017
2. Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on Hierarchical and grouped me series 2
3. Labour market par cipa on Australia and New Zealand Standard Classifica on of Occupa ons 8 major groups 43 sub-major groups 97 minor groups – 359 unit groups * 1023 occupa ons Example: Sta s cian 2 Professionals 22 Business, Human Resource and Marke ng Professionals 224 Informa on and Organisa on Professionals 2241 Actuaries, Mathema cians and Sta s cians 224113 Sta s cian Op mal Forecast Reconcilia on Hierarchical and grouped me series 3
4. Labour market par cipa on Australia and New Zealand Standard Classifica on of Occupa ons 8 major groups 43 sub-major groups 97 minor groups – 359 unit groups * 1023 occupa ons Example: Sta s cian 2 Professionals 22 Business, Human Resource and Marke ng Professionals 224 Informa on and Organisa on Professionals 2241 Actuaries, Mathema cians and Sta s cians 224113 Sta s cian Op mal Forecast Reconcilia on Hierarchical and grouped me series 3
5. Australian tourism demand Op mal Forecast Reconcilia on Hierarchical and grouped me series 4
6. Australian tourism demand Op mal Forecast Reconcilia on Hierarchical and grouped me series 4 Quarterly data on visitor night from 1998:Q1 – 2013:Q4 From Na onal Visitor Survey, based on annual interviews of 120,000 Australians aged 15+, collected by Tourism Research Australia. Split by 7 states, 27 zones and 76 regions (a geographical hierarchy) Also split by purpose of travel Holiday Visi ng friends and rela ves (VFR) Business Other 304 bo om-level series
7. Spectacle sales Op mal Forecast Reconcilia on Hierarchical and grouped me series 5 Monthly UK sales data from 2000 – 2014 Provided by a large spectacle manufacturer Split by brand (26), gender (3), price range (6), materials (4), and stores (600) About 1 million bo om-level series
8. Spectacle sales Op mal Forecast Reconcilia on Hierarchical and grouped me series 5 Monthly UK sales data from 2000 – 2014 Provided by a large spectacle manufacturer Split by brand (26), gender (3), price range (6), materials (4), and stores (600) About 1 million bo om-level series
9. Spectacle sales Op mal Forecast Reconcilia on Hierarchical and grouped me series 5 Monthly UK sales data from 2000 – 2014 Provided by a large spectacle manufacturer Split by brand (26), gender (3), price range (6), materials (4), and stores (600) About 1 million bo om-level series
10. Spectacle sales Op mal Forecast Reconcilia on Hierarchical and grouped me series 5 Monthly UK sales data from 2000 – 2014 Provided by a large spectacle manufacturer Split by brand (26), gender (3), price range (6), materials (4), and stores (600) About 1 million bo om-level series
11. Hierarchical me series A hierarchical me series is a collec on of several me series that are linked together in a hierarchical structure. Total A AA AB AC B BA BB BC C CA CB CC Examples Labour turnover by occupa on Tourism by state and region Op mal Forecast Reconcilia on Hierarchical and grouped me series 6
12. Hierarchical me series A hierarchical me series is a collec on of several me series that are linked together in a hierarchical structure. Total A AA AB AC B BA BB BC C CA CB CC Examples Labour turnover by occupa on Tourism by state and region Op mal Forecast Reconcilia on Hierarchical and grouped me series 6
13. Hierarchical me series A hierarchical me series is a collec on of several me series that are linked together in a hierarchical structure. Total A AA AB AC B BA BB BC C CA CB CC Examples Labour turnover by occupa on Tourism by state and region Op mal Forecast Reconcilia on Hierarchical and grouped me series 6
14. Grouped me series A grouped me series is a collec on of me series that can be grouped together in a number of non-hierarchical ways. Total A AX AY B BX BY Total X AX BX Y AY BY Examples Labour turnover by occupa on and state Tourism by region and purpose of travel Spectacle sales by brand, gender, stores, etc. Op mal Forecast Reconcilia on Hierarchical and grouped me series 7
15. Grouped me series A grouped me series is a collec on of me series that can be grouped together in a number of non-hierarchical ways. Total A AX AY B BX BY Total X AX BX Y AY BY Examples Labour turnover by occupa on and state Tourism by region and purpose of travel Spectacle sales by brand, gender, stores, etc. Op mal Forecast Reconcilia on Hierarchical and grouped me series 7
16. Grouped me series A grouped me series is a collec on of me series that can be grouped together in a number of non-hierarchical ways. Total A AX AY B BX BY Total X AX BX Y AY BY Examples Labour turnover by occupa on and state Tourism by region and purpose of travel Spectacle sales by brand, gender, stores, etc. Op mal Forecast Reconcilia on Hierarchical and grouped me series 7
17. Grouped me series A grouped me series is a collec on of me series that can be grouped together in a number of non-hierarchical ways. Total A AX AY B BX BY Total X AX BX Y AY BY Examples Labour turnover by occupa on and state Tourism by region and purpose of travel Spectacle sales by brand, gender, stores, etc. Op mal Forecast Reconcilia on Hierarchical and grouped me series 7
18. tl;dr 1 Forecast all series at all levels of aggrega on using an automa c forecas ng algorithm (e.g., ets, auto.arima, ...) 2 Reconcile the resul ng forecasts so they add up correctly using least squares op miza on (i.e., find closest reconciled forecasts to the original forecasts). 3 This is all available in the hts package in R. Op mal Forecast Reconcilia on Hierarchical and grouped me series 8
19. tl;dr 1 Forecast all series at all levels of aggrega on using an automa c forecas ng algorithm (e.g., ets, auto.arima, ...) 2 Reconcile the resul ng forecasts so they add up correctly using least squares op miza on (i.e., find closest reconciled forecasts to the original forecasts). 3 This is all available in the hts package in R. Op mal Forecast Reconcilia on Hierarchical and grouped me series 8
20. tl;dr 1 Forecast all series at all levels of aggrega on using an automa c forecas ng algorithm (e.g., ets, auto.arima, ...) 2 Reconcile the resul ng forecasts so they add up correctly using least squares op miza on (i.e., find closest reconciled forecasts to the original forecasts). 3 This is all available in the hts package in R. Op mal Forecast Reconcilia on Hierarchical and grouped me series 8
21. Hierarchical me series Total A B C Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.
22. Hierarchical me series Total A B C Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.
23. Hierarchical me series Total A B C yt = [yt , yA,t , yB,t , yC,t ] =     1 1 1 1 0 0 0 1 0 0 0 1       yA,t yB,t yC,t   Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.
24. Hierarchical me series Total A B C yt = [yt , yA,t , yB,t , yC,t ] =     1 1 1 1 0 0 0 1 0 0 0 1     S   yA,t yB,t yC,t   Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.
25. Hierarchical me series Total A B C yt = [yt , yA,t , yB,t , yC,t ] =     1 1 1 1 0 0 0 1 0 0 0 1     S   yA,t yB,t yC,t   bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.
26. Hierarchical me series Total A B C yt = [yt , yA,t , yB,t , yC,t ] =     1 1 1 1 0 0 0 1 0 0 0 1     S   yA,t yB,t yC,t   bt yt = Sbt Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.
27. Hierarchical me series Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yB,t yC,t yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 10
28. Hierarchical me series Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yB,t yC,t yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 10
29. Hierarchical me series Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yB,t yC,t yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 10 yt = Sbt
30. Grouped data AX AY A BX BY B X Y Total yt =             yt yA,t yB,t yX,t yY,t yAX,t yAY,t yBX,t yBY,t             =             1 1 1 1 1 1 0 0 0 0 1 1 1 0 1 0 0 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1             S    yAX,t yAY,t yBX,t yBY,t    bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 11
31. Grouped data AX AY A BX BY B X Y Total yt =             yt yA,t yB,t yX,t yY,t yAX,t yAY,t yBX,t yBY,t             =             1 1 1 1 1 1 0 0 0 0 1 1 1 0 1 0 0 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1             S    yAX,t yAY,t yBX,t yBY,t    bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 11
32. Grouped data AX AY A BX BY B X Y Total yt =             yt yA,t yB,t yX,t yY,t yAX,t yAY,t yBX,t yBY,t             =             1 1 1 1 1 1 0 0 0 0 1 1 1 0 1 0 0 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1             S    yAX,t yAY,t yBX,t yBY,t    bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 11 yt = Sbt
33. Hierarchical and grouped me series Every collec on of me series with aggrega on constraints can be wri en as yt = Sbt where yt is a vector of all series at me t bt is a vector of the most disaggregated series at me t S is a “summing matrix” containing the aggrega on constraints. Op mal Forecast Reconcilia on Hierarchical and grouped me series 12
34. Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on Minimum trace reconcilia on 13
35. Forecas ng nota on Let ˆ yn (h) be vector of ini al h-step forecasts, made at me n, stacked in same order as yt. (In general, they will not “add up”.) Reconciled forecasts must be of the form: ˜ yn (h) = SPˆ yn (h) for some matrix P. P extracts and combines base forecasts ˆ yn (h) to get bo om-level forecasts. S adds them up Op mal Forecast Reconcilia on Minimum trace reconcilia on 14
36. Forecas ng nota on Let ˆ yn (h) be vector of ini al h-step forecasts, made at me n, stacked in same order as yt. (In general, they will not “add up”.) Reconciled forecasts must be of the form: ˜ yn (h) = SPˆ yn (h) for some matrix P. P extracts and combines base forecasts ˆ yn (h) to get bo om-level forecasts. S adds them up Op mal Forecast Reconcilia on Minimum trace reconcilia on 14
37. Forecas ng nota on Let ˆ yn (h) be vector of ini al h-step forecasts, made at me n, stacked in same order as yt. (In general, they will not “add up”.) Reconciled forecasts must be of the form: ˜ yn (h) = SPˆ yn (h) for some matrix P. P extracts and combines base forecasts ˆ yn (h) to get bo om-level forecasts. S adds them up Op mal Forecast Reconcilia on Minimum trace reconcilia on 14
38. Forecas ng nota on Let ˆ yn (h) be vector of ini al h-step forecasts, made at me n, stacked in same order as yt. (In general, they will not “add up”.) Reconciled forecasts must be of the form: ˜ yn (h) = SPˆ yn (h) for some matrix P. P extracts and combines base forecasts ˆ yn (h) to get bo om-level forecasts. S adds them up Op mal Forecast Reconcilia on Minimum trace reconcilia on 14
39. Forecas ng nota on Let ˆ yn (h) be vector of ini al h-step forecasts, made at me n, stacked in same order as yt. (In general, they will not “add up”.) Reconciled forecasts must be of the form: ˜ yn (h) = SPˆ yn (h) for some matrix P. P extracts and combines base forecasts ˆ yn (h) to get bo om-level forecasts. S adds them up Op mal Forecast Reconcilia on Minimum trace reconcilia on 14
40. General proper es: bias and variance ˜ yn (h) = SPˆ yn (h) Bias Reconciled forecasts are unbiased iff SPS = S. Variance Let error variance of h-step base forecasts ˆ yn (h) be Wh = Var[yn+h − ˆ yn (h) | y1 , . . . , yn ] Then the error variance of the corresponding reconciled forecasts is Var[yn+h − ˜ yn (h) | y1 , . . . , yn ] = SPWh P S Op mal Forecast Reconcilia on Minimum trace reconcilia on 15
41. General proper es: bias and variance ˜ yn (h) = SPˆ yn (h) Bias Reconciled forecasts are unbiased iff SPS = S. Variance Let error variance of h-step base forecasts ˆ yn (h) be Wh = Var[yn+h − ˆ yn (h) | y1 , . . . , yn ] Then the error variance of the corresponding reconciled forecasts is Var[yn+h − ˜ yn (h) | y1 , . . . , yn ] = SPWh P S Op mal Forecast Reconcilia on Minimum trace reconcilia on 15
42. General proper es: bias and variance ˜ yn (h) = SPˆ yn (h) Bias Reconciled forecasts are unbiased iff SPS = S. Variance Let error variance of h-step base forecasts ˆ yn (h) be Wh = Var[yn+h − ˆ yn (h) | y1 , . . . , yn ] Then the error variance of the corresponding reconciled forecasts is Var[yn+h − ˜ yn (h) | y1 , . . . , yn ] = SPWh P S Op mal Forecast Reconcilia on Minimum trace reconcilia on 15
43. Op mal forecast reconcilia on ˜ yn (h) = SPˆ yn (h) Theorem: MinT Reconcilia on If P sa sfies SPS = S, then minP = trace[SPWh P S ] has solu on P = (S W−1 h S)−1S W−1 h . Reconciled forecasts Base forecasts Assume that Wh = kh W1 to simplify computa ons. Op mal Forecast Reconcilia on Minimum trace reconcilia on 16 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
44. Op mal forecast reconcilia on ˜ yn (h) = SPˆ yn (h) Theorem: MinT Reconcilia on If P sa sfies SPS = S, then minP = trace[SPWh P S ] has solu on P = (S W−1 h S)−1S W−1 h . Reconciled forecasts Base forecasts Assume that Wh = kh W1 to simplify computa ons. Op mal Forecast Reconcilia on Minimum trace reconcilia on 16 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
45. Op mal forecast reconcilia on ˜ yn (h) = SPˆ yn (h) Theorem: MinT Reconcilia on If P sa sfies SPS = S, then minP = trace[SPWh P S ] has solu on P = (S W−1 h S)−1S W−1 h . Reconciled forecasts Base forecasts Assume that Wh = kh W1 to simplify computa ons. Op mal Forecast Reconcilia on Minimum trace reconcilia on 16 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
46. Op mal forecast reconcilia on ˜ yn (h) = SPˆ yn (h) Theorem: MinT Reconcilia on If P sa sfies SPS = S, then minP = trace[SPWh P S ] has solu on P = (S W−1 h S)−1S W−1 h . Reconciled forecasts Base forecasts Assume that Wh = kh W1 to simplify computa ons. Op mal Forecast Reconcilia on Minimum trace reconcilia on 16 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
47. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)
48. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)
49. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)
50. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)
51. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)
52. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)
53. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 2: WLS Approximate W1 by its diagonal. Easy to es mate, and places weight where we have best forecasts. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 18 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
54. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 2: WLS Approximate W1 by its diagonal. Easy to es mate, and places weight where we have best forecasts. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 18 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
55. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 2: WLS Approximate W1 by its diagonal. Easy to es mate, and places weight where we have best forecasts. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 18 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
56. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 2: WLS Approximate W1 by its diagonal. Easy to es mate, and places weight where we have best forecasts. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 18 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
57. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 3: GLS Es mate W1 using shrinkage to the diagonal. Allows for covariances. Difficult to compute for large numbers of me series. Op mal Forecast Reconcilia on Minimum trace reconcilia on 19 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
58. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 3: GLS Es mate W1 using shrinkage to the diagonal. Allows for covariances. Difficult to compute for large numbers of me series. Op mal Forecast Reconcilia on Minimum trace reconcilia on 19 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
59. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 3: GLS Es mate W1 using shrinkage to the diagonal. Allows for covariances. Difficult to compute for large numbers of me series. Op mal Forecast Reconcilia on Minimum trace reconcilia on 19 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
60. Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 3: GLS Es mate W1 using shrinkage to the diagonal. Allows for covariances. Difficult to compute for large numbers of me series. Op mal Forecast Reconcilia on Minimum trace reconcilia on 19 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)
61. Australian tourism Op mal Forecast Reconcilia on Minimum trace reconcilia on 20
62. Australian tourism Op mal Forecast Reconcilia on Minimum trace reconcilia on 20 Hierarchy: States (7) Zones (27) Regions (82)
63. Australian tourism Op mal Forecast Reconcilia on Minimum trace reconcilia on 20 Hierarchy: States (7) Zones (27) Regions (82) Base forecasts ETS (exponen al smoothing) models
64. Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: Total Year Visitor nights 1998 2000 2002 2004 2006 2008 60000 65000 70000 75000 80000 85000
65. Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: NSW Year Visitor nights 1998 2000 2002 2004 2006 2008 18000 22000 26000 30000
66. Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: VIC Year Visitor nights 1998 2000 2002 2004 2006 2008 10000 12000 14000 16000 18000
67. Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: Nth.Coast.NSW Year Visitor nights 1998 2000 2002 2004 2006 2008 5000 6000 7000 8000 9000
68. Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: Metro.QLD Year Visitor nights 1998 2000 2002 2004 2006 2008 8000 9000 11000 13000
69. Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: Sth.WA Year Visitor nights 1998 2000 2002 2004 2006 2008 400 600 800 1000 1200 1400
70. Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: X201.Melbourne Year Visitor nights 1998 2000 2002 2004 2006 2008 4000 4500 5000 5500 6000
71. Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: X402.Murraylands Year Visitor nights 1998 2000 2002 2004 2006 2008 0 100 200 300
72. Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: X809.Daly Year Visitor nights 1998 2000 2002 2004 2006 2008 0 20 40 60 80 100
73. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q time
74. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
75. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
76. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
77. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
78. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
79. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
80. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
81. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
82. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
83. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
84. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
85. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
86. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
87. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
88. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
89. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
90. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
91. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
92. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
93. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 2 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
94. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 3 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
95. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 4 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
96. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 5 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
97. Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 6 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time
98. Hierarchy: states, zones, regions Forecast horizon RMSE h = 1 h = 2 h = 3 h = 4 h = 5 h = 6 Ave Australia Base 1762.04 1770.29 1766.02 1818.82 1705.35 1721.17 1757.28 Bo om 1736.92 1742.69 1722.79 1752.74 1666.73 1687.43 1718.22 OLS 1747.60 1757.68 1751.77 1800.67 1686.00 1706.45 1741.69 WLS 1705.21 1715.87 1703.75 1729.56 1627.79 1661.24 1690.57 GLS 1704.64 1715.60 1705.31 1729.04 1626.36 1661.64 1690.43 States Base 399.77 404.16 401.92 407.26 395.38 401.17 401.61 Bo om 404.29 406.95 404.96 409.02 399.80 401.55 404.43 OLS 404.47 407.62 405.43 413.79 401.10 404.90 406.22 WLS 398.84 402.12 400.71 405.03 394.76 398.23 399.95 GLS 398.84 402.16 400.86 405.03 394.59 398.22 399.95 Regions Base 93.15 93.38 93.45 93.79 93.50 93.56 93.47 Bo om 93.15 93.38 93.45 93.79 93.50 93.56 93.47 OLS 93.28 93.53 93.64 94.17 93.78 93.88 93.71 WLS 93.02 93.32 93.38 93.72 93.39 93.53 93.39 GLS 92.98 93.27 93.34 93.66 93.34 93.46 93.34 Op mal Forecast Reconcilia on Minimum trace reconcilia on 23
99. Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on Fast computa onal tricks 24
100. Fast computa on: hierarchical data Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yB,t yC,t yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Fast computa onal tricks 25 yt = Sbt
101. Fast computa on: hierarchical data Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yAX,t yAY,t yAZ,t yB,t yBX,t yBY,t yBZ,t yC,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Fast computa onal tricks 26 yt = Sbt
102. Fast computa on: hierarchical data Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yAX,t yAY,t yAZ,t yB,t yBX,t yBY,t yBZ,t yC,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Fast computa onal tricks 26 yt = Sbt
103. Fast computa on: hierarchies Think of the hierarchy as a tree of trees: Total T1 T2 ... TK Then the summing matrix contains k smaller summing matrices: S =       1n1 1n2 · · · 1nK S1 0 · · · 0 0 S2 · · · 0 . . . . . . ... . . . 0 0 · · · SK       where 1n is an n-vector of ones and tree Ti has ni terminal nodes. Op mal Forecast Reconcilia on Fast computa onal tricks 27
104. Fast computa on: hierarchies Think of the hierarchy as a tree of trees: Total T1 T2 ... TK Then the summing matrix contains k smaller summing matrices: S =       1n1 1n2 · · · 1nK S1 0 · · · 0 0 S2 · · · 0 . . . . . . ... . . . 0 0 · · · SK       where 1n is an n-vector of ones and tree Ti has ni terminal nodes. Op mal Forecast Reconcilia on Fast computa onal tricks 27
105. Fast computa on: hierarchies SΛS =     S1 Λ1 S1 0 · · · 0 0 S2 Λ2 S2 · · · 0 . . . . . . ... . . . 0 0 · · · SK ΛK SK     + λ0 Jn λ0 is the top le element of Λ; Λk is a block of Λ, corresponding to tree Tk; Jn is a matrix of ones; n = k nk. Now apply the Sherman-Morrison formula ... Op mal Forecast Reconcilia on Fast computa onal tricks 28
106. Fast computa on: hierarchies SΛS =     S1 Λ1 S1 0 · · · 0 0 S2 Λ2 S2 · · · 0 . . . . . . ... . . . 0 0 · · · SK ΛK SK     + λ0 Jn λ0 is the top le element of Λ; Λk is a block of Λ, corresponding to tree Tk; Jn is a matrix of ones; n = k nk. Now apply the Sherman-Morrison formula ... Op mal Forecast Reconcilia on Fast computa onal tricks 28
107. Fast computa on: hierarchies (S ΛS)−1 =     (S1 Λ1 S1 )−1 0 · · · 0 0 (S2 Λ2 S2 )−1 · · · 0 . . . . . . ... . . . 0 0 · · · (SK ΛK SK )−1     − cS0 S0 can be par oned into K2 blocks, with the (k, ) block (of dimension nk × n ) being (Sk Λk Sk )−1Jnk ,n (S Λ S )−1 Jnk ,n is a nk × n matrix of ones. c−1 = λ−1 0 + k 1nk (Sk Λk Sk )−11nk . Each Sk Λk Sk can be inverted similarly. S Λy can also be computed recursively. Op mal Forecast Reconcilia on Fast computa onal tricks 29
108. Fast computa on: hierarchies (S ΛS)−1 =     (S1 Λ1 S1 )−1 0 · · · 0 0 (S2 Λ2 S2 )−1 · · · 0 . . . . . . ... . . . 0 0 · · · (SK ΛK SK )−1     − cS0 S0 can be par oned into K2 blocks, with the (k, ) block (of dimension nk × n ) being (Sk Λk Sk )−1Jnk ,n (S Λ S )−1 Jnk ,n is a nk × n matrix of ones. c−1 = λ−1 0 + k 1nk (Sk Λk Sk )−11nk . Each Sk Λk Sk can be inverted similarly. S Λy can also be computed recursively. Op mal Forecast Reconcilia on Fast computa onal tricks 29 The recursive calcula ons can be done in such a way that we never store any of the large matrices involved.
109. Fast computa on A similar algorithm has been developed for grouped me series with two groups. When the me series are not strictly hierarchical and have more than two grouping variables: Use sparse matrix storage and arithme c. Use itera ve approxima on for inver ng large sparse matrices. Paige & Saunders (1982) ACM Trans. Math. So ware Op mal Forecast Reconcilia on Fast computa onal tricks 30
110. Fast computa on A similar algorithm has been developed for grouped me series with two groups. When the me series are not strictly hierarchical and have more than two grouping variables: Use sparse matrix storage and arithme c. Use itera ve approxima on for inver ng large sparse matrices. Paige & Saunders (1982) ACM Trans. Math. So ware Op mal Forecast Reconcilia on Fast computa onal tricks 30
111. Fast computa on A similar algorithm has been developed for grouped me series with two groups. When the me series are not strictly hierarchical and have more than two grouping variables: Use sparse matrix storage and arithme c. Use itera ve approxima on for inver ng large sparse matrices. Paige & Saunders (1982) ACM Trans. Math. So ware Op mal Forecast Reconcilia on Fast computa onal tricks 30
112. Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on Temporal hierarchies 31
113. Temporal hierarchies Annual Semi-Annual1 Q1 Q2 Semi-Annual2 Q3 Q4 Basic idea: ¯ Forecast series at each available frequency. ¯ Op mally reconcile forecasts within the same year. Op mal Forecast Reconcilia on Temporal hierarchies 32
114. Temporal hierarchies Annual Semi-Annual1 Q1 Q2 Semi-Annual2 Q3 Q4 Basic idea: ¯ Forecast series at each available frequency. ¯ Op mally reconcile forecasts within the same year. Op mal Forecast Reconcilia on Temporal hierarchies 32
115. Monthly series Annual Semi-Annual1 Q1 M1 M2 M3 Q2 M4 M5 M6 Semi-Annual2 Q3 M7 M8 M9 Q4 M10 M11 M12 k = 2, 4, 12 nodes k = 3, 6, 12 nodes Why not k = 2, 3, 4, 6, 12 nodes? Op mal Forecast Reconcilia on Temporal hierarchies 33
116. Monthly series Annual FourM1 BiM1 M1 M2 BiM2 M3 M4 FourM2 BiM3 M5 M6 BiM4 M7 M8 FourM3 BiM5 M9 M10 BiM6 M11 M12 k = 2, 4, 12 nodes k = 3, 6, 12 nodes Why not k = 2, 3, 4, 6, 12 nodes? Op mal Forecast Reconcilia on Temporal hierarchies 33
117. Monthly series Annual FourM1 BiM1 M1 M2 BiM2 M3 M4 FourM2 BiM3 M5 M6 BiM4 M7 M8 FourM3 BiM5 M9 M10 BiM6 M11 M12 k = 2, 4, 12 nodes k = 3, 6, 12 nodes Why not k = 2, 3, 4, 6, 12 nodes? Op mal Forecast Reconcilia on Temporal hierarchies 33
118. Monthly data                   A SemiA1 SemiA2 FourM1 FourM2 FourM3 Q1 . . . Q4 BiM1 . . . BiM6 M1 . . . M12                   (28×1) =                   1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 . . . 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 . . . 0 0 0 0 0 0 0 0 0 0 1 1 I12                   S                M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12                Bt Op mal Forecast Reconcilia on Temporal hierarchies 34
119. In general For a me series y1 , . . . , yT, observed at frequency m, we generate aggregate series y[k] j = jk t=1+(j−1)k yt , for j = 1, . . . , T/k k ∈ F(m) = {factors of m}. A single unique hierarchy is only possible when there are no coprime pairs in F(m). Mk = m/k is seasonal period of aggregated series. Op mal Forecast Reconcilia on Temporal hierarchies 35
120. In general For a me series y1 , . . . , yT, observed at frequency m, we generate aggregate series y[k] j = jk t=1+(j−1)k yt , for j = 1, . . . , T/k k ∈ F(m) = {factors of m}. A single unique hierarchy is only possible when there are no coprime pairs in F(m). Mk = m/k is seasonal period of aggregated series. Op mal Forecast Reconcilia on Temporal hierarchies 35
121. In general For a me series y1 , . . . , yT, observed at frequency m, we generate aggregate series y[k] j = jk t=1+(j−1)k yt , for j = 1, . . . , T/k k ∈ F(m) = {factors of m}. A single unique hierarchy is only possible when there are no coprime pairs in F(m). Mk = m/k is seasonal period of aggregated series. Op mal Forecast Reconcilia on Temporal hierarchies 35
122. WLS weights Hierarchy variance scaling ΛH: diagonal. Series variance scaling ΛV: elements equal within aggrega on level. Structural scaling ΛS = diag(S1): elements equal to # nodes at each level. Depends only on seasonal period m. Independent of data and model. Allows forecasts where no errors available. Quarterly example ΛH = diag ˆ σ2 A , ˆ σ2 S1 , ˆ σ2 S2 , ˆ σ2 Q1 , ˆ σ2 Q2 , ˆ σ2 Q3 , ˆ σ2 Q4 ΛV = diag ˆ σ2 A , ˆ σ2 S , ˆ σ2 S , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q ΛS = diag 4, 2, 2, 1, 1, 1, 1 Op mal Forecast Reconcilia on Temporal hierarchies 36
123. WLS weights Hierarchy variance scaling ΛH: diagonal. Series variance scaling ΛV: elements equal within aggrega on level. Structural scaling ΛS = diag(S1): elements equal to # nodes at each level. Depends only on seasonal period m. Independent of data and model. Allows forecasts where no errors available. Quarterly example ΛH = diag ˆ σ2 A , ˆ σ2 S1 , ˆ σ2 S2 , ˆ σ2 Q1 , ˆ σ2 Q2 , ˆ σ2 Q3 , ˆ σ2 Q4 ΛV = diag ˆ σ2 A , ˆ σ2 S , ˆ σ2 S , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q ΛS = diag 4, 2, 2, 1, 1, 1, 1 Op mal Forecast Reconcilia on Temporal hierarchies 36
124. WLS weights Hierarchy variance scaling ΛH: diagonal. Series variance scaling ΛV: elements equal within aggrega on level. Structural scaling ΛS = diag(S1): elements equal to # nodes at each level. Depends only on seasonal period m. Independent of data and model. Allows forecasts where no errors available. Quarterly example ΛH = diag ˆ σ2 A , ˆ σ2 S1 , ˆ σ2 S2 , ˆ σ2 Q1 , ˆ σ2 Q2 , ˆ σ2 Q3 , ˆ σ2 Q4 ΛV = diag ˆ σ2 A , ˆ σ2 S , ˆ σ2 S , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q ΛS = diag 4, 2, 2, 1, 1, 1, 1 Op mal Forecast Reconcilia on Temporal hierarchies 36
125. UK Accidents and Emergency Demand Op mal Forecast Reconcilia on Temporal hierarchies 37 1 2 3 4 5 6 5100 5300 5500 Annual (k=52) Forecast 2 4 6 8 10 12 2500 2600 2700 2800 2900 Semi−annual (k=26) Forecast 5 10 15 20 25 1250 1350 1450 Quarterly (k=13) Forecast 20 40 60 80 360 380 400 420 440 460 Monthly (k=4) Forecast 50 100 150 180 190 200 210 220 230 Bi−weekly (k=2) Forecast 50 100 150 200 250 300 90 95 100 105 110 Weekly (k=1) Forecast – – – – base reconciled
126. UK Accidents and Emergency Demand 1 Type 1 Departments — Major A&E 2 Type 2 Departments — Single Specialty 3 Type 3 Departments — Other A&E/Minor Injury 4 Total A endances 5 Type 1 Departments — Major A&E > 4 hrs 6 Type 2 Departments — Single Specialty > 4 hrs 7 Type 3 Departments — Other A&E/Minor Injury > 4 hrs 8 Total A endances > 4 hrs 9 Emergency Admissions via Type 1 A&E 10 Total Emergency Admissions via A&E 11 Other Emergency Admissions (i.e., not via A&E) 12 Total Emergency Admissions 13 Number of pa ents spending > 4 hrs from decision to admission Op mal Forecast Reconcilia on Temporal hierarchies 38
127. UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39
128. UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39
129. UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39
130. UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39
131. UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39
132. UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39
133. Experimental setup: M3 forecas ng compe on (Makridakis and Hibon, 2000, IJF). In total 3003 series. 1,428 monthly series with a test sample of 12 observa ons each. 756 quarterly series with a test sample of 8 observa ons each. Forecast each series with ETS models. Op mal Forecast Reconcilia on Temporal hierarchies 40
134. Experimental setup: M3 forecas ng compe on (Makridakis and Hibon, 2000, IJF). In total 3003 series. 1,428 monthly series with a test sample of 12 observa ons each. 756 quarterly series with a test sample of 8 observa ons each. Forecast each series with ETS models. Op mal Forecast Reconcilia on Temporal hierarchies 40
135. Experimental setup: M3 forecas ng compe on (Makridakis and Hibon, 2000, IJF). In total 3003 series. 1,428 monthly series with a test sample of 12 observa ons each. 756 quarterly series with a test sample of 8 observa ons each. Forecast each series with ETS models. Op mal Forecast Reconcilia on Temporal hierarchies 40
136. Experimental setup: M3 forecas ng compe on (Makridakis and Hibon, 2000, IJF). In total 3003 series. 1,428 monthly series with a test sample of 12 observa ons each. 756 quarterly series with a test sample of 8 observa ons each. Forecast each series with ETS models. Op mal Forecast Reconcilia on Temporal hierarchies 40
137. Results: Monthly MAE percent difference rela ve to base max h BU WLSH WLSV WLSS Annual 1 −19.6 −22.0 −22.0 −25.1 Semi-annual 3 0.6 −4.0 −3.6 −5.4 Four-monthly 4 2.0 −2.4 −2.2 −3.0 Quarterly 6 2.4 −1.6 −1.7 −2.8 Bi-monthly 9 0.7 −2.9 −3.3 −4.3 Monthly 18 0.0 −2.2 −3.2 −3.9 Op mal Forecast Reconcilia on Temporal hierarchies 41
138. Results: Quarterly MAE percent difference rela ve to base max h BU WLSH WLSV WLSS Annual 1 −20.9 -22.7 −22.8 -22.7 Semi-annual 3 −4.5 −6.0 −6.2 -4.8 Quarterly 6 0.0 −0.2 −1.1 -0.3 Op mal Forecast Reconcilia on Temporal hierarchies 42
139. Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on Probabilis c reconcilia on 43
140. Coherent density forecasts Defini on: Coherence Suppose yt ∈ Rn. yt is coherent if yt lies in an m-dimensional subspace of Rn spanned by the columns of the summing matrix S. Defini on: Coherent density forecasts Any density p(yt+h ) such that p(yt+h ) = 0 for all yt+h in the null space of S. Coherent point forecasts: ˜ yT+h|T = SPˆ yT+h. Coherent variance forecasts (assuming unbiasedness): ˜ ΣT+h = SPˆ ΣT+h P S Op mal Forecast Reconcilia on Probabilis c reconcilia on 44
141. Coherent density forecasts Defini on: Coherence Suppose yt ∈ Rn. yt is coherent if yt lies in an m-dimensional subspace of Rn spanned by the columns of the summing matrix S. Defini on: Coherent density forecasts Any density p(yt+h ) such that p(yt+h ) = 0 for all yt+h in the null space of S. Coherent point forecasts: ˜ yT+h|T = SPˆ yT+h. Coherent variance forecasts (assuming unbiasedness): ˜ ΣT+h = SPˆ ΣT+h P S Op mal Forecast Reconcilia on Probabilis c reconcilia on 44
142. Coherent Gaussian forecasts yT+h|T ∼ N(˜ yT+h|T , ˜ ΣT+h ) Let L be the Energy Score (a proper scoring rule): L(˜ YT+h , yT+h ) = E ˜ YT+h − yT+h α − 1 2 E ˜ YT+h − ˜ YT+h α for α ∈ (0, 2], where ˜ YT+h and ˜ YT+h are independent rvs from N(˜ yT+h|T , ˜ ΣT+h ). There is no closed form expression for L(˜ YT+h , yT+h ) for α ∈ (0, 2) under the Gaussian predic ve distribu on. When α = 2, L(˜ YT+h , yT+h ) = ˜ YT+h − yT+h 2 This is equivalent to MinT solu on. Op mal Forecast Reconcilia on Probabilis c reconcilia on 45
143. Coherent nonparametric forecasts 1 Simulate forecast distribu ons at each bo om level node using univariate models 2 Compute empirical copulas for each parent+children group. Equivalent to “bo om-up” point forecas ng. No reconcilia on involved. Successfully applied to forecas ng smart-metre electricity demand (Ben Taieb, Taylor, Hyndman, 2017) Op mal Forecast Reconcilia on Probabilis c reconcilia on 46
144. Coherent nonparametric forecasts 1 Simulate forecast distribu ons at each bo om level node using univariate models 2 Compute empirical copulas for each parent+children group. Equivalent to “bo om-up” point forecas ng. No reconcilia on involved. Successfully applied to forecas ng smart-metre electricity demand (Ben Taieb, Taylor, Hyndman, 2017) Op mal Forecast Reconcilia on Probabilis c reconcilia on 46
145. Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on References 47
146. References RJ Hyndman, RA Ahmed, G Athanasopoulos, and HL Shang (2011). Op mal combina on forecasts for hierarchical me series. Computa onal Sta s cs & Data Analysis 55(9), 2579–2589. RJ Hyndman, A Lee, and E Wang (2016). Fast computa on of reconciled forecasts for hierarchical and grouped me series. Computa onal Sta s cs & Data Analysis 97, 16–32 SL Wickramasuriya, G Athanasopoulos, and RJ Hyndman (2015). Forecas ng hierarchical and grouped me series through trace minimiza on. Working paper. Dept Econometrics & Business Sta s cs, Monash University G Athanasopoulos, RJ Hyndman, N Kourentzes, and F Petropoulos (2017). Forecas ng with temporal hierarchies. European Journal of Opera onal Research 262(1), 60–74 S Ben Taieb, JW Taylor, and RJ Hyndman (2017). Hierarchical Probabilis c Forecas ng of Electricity Demand with Smart Meter Data. Working paper. Dept Econometrics & Business Sta s cs, Monash University RJ Hyndman, A Lee, E Wang, and S Wickramasuriya (2017). hts: Hierarchical and Grouped Time Series. Version 5.1.4. https://CRAN.R-project.org/package=hts RJ Hyndman and N Kourentzes (2016). thief: Temporal Hierarchical Forecas ng. Version 0.2. https://CRAN.R-project.org/package=thief Op mal Forecast Reconcilia on References 48
147. References RJ Hyndman, RA Ahmed, G Athanasopoulos, and HL Shang (2011). Op mal combina on forecasts for hierarchical me series. Computa onal Sta s cs & Data Analysis 55(9), 2579–2589. RJ Hyndman, A Lee, and E Wang (2016). Fast computa on of reconciled forecasts for hierarchical and grouped me series. Computa onal Sta s cs & Data Analysis 97, 16–32 SL Wickramasuriya, G Athanasopoulos, and RJ Hyndman (2015). Forecas ng hierarchical and grouped me series through trace minimiza on. Working paper. Dept Econometrics & Business Sta s cs, Monash University G Athanasopoulos, RJ Hyndman, N Kourentzes, and F Petropoulos (2017). Forecas ng with temporal hierarchies. European Journal of Opera onal Research 262(1), 60–74 S Ben Taieb, JW Taylor, and RJ Hyndman (2017). Hierarchical Probabilis c Forecas ng of Electricity Demand with Smart Meter Data. Working paper. Dept Econometrics & Business Sta s cs, Monash University RJ Hyndman, A Lee, E Wang, and S Wickramasuriya (2017). hts: Hierarchical and Grouped Time Series. Version 5.1.4. https://CRAN.R-project.org/package=hts RJ Hyndman and N Kourentzes (2016). thief: Temporal Hierarchical Forecas ng. Version 0.2. https://CRAN.R-project.org/package=thief Op mal Forecast Reconcilia on References 48 ¯ More informa on: robjhyndman.com