Tweet
Tweet

Slide 1

Slide 1 text

Op mal Forecast Reconcilia on Rob J Hyndman UNSW: 25 August 2017

Slide 2

Slide 2 text

Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on Hierarchical and grouped me series 2

Slide 3

Slide 3 text

Labour market par cipa on Australia and New Zealand Standard Classifica on of Occupa ons 8 major groups 43 sub-major groups 97 minor groups – 359 unit groups * 1023 occupa ons Example: Sta s cian 2 Professionals 22 Business, Human Resource and Marke ng Professionals 224 Informa on and Organisa on Professionals 2241 Actuaries, Mathema cians and Sta s cians 224113 Sta s cian Op mal Forecast Reconcilia on Hierarchical and grouped me series 3

Slide 4

Slide 4 text

Labour market par cipa on Australia and New Zealand Standard Classifica on of Occupa ons 8 major groups 43 sub-major groups 97 minor groups – 359 unit groups * 1023 occupa ons Example: Sta s cian 2 Professionals 22 Business, Human Resource and Marke ng Professionals 224 Informa on and Organisa on Professionals 2241 Actuaries, Mathema cians and Sta s cians 224113 Sta s cian Op mal Forecast Reconcilia on Hierarchical and grouped me series 3

Slide 5

Slide 5 text

Australian tourism demand Op mal Forecast Reconcilia on Hierarchical and grouped me series 4

Slide 6

Slide 6 text

Australian tourism demand Op mal Forecast Reconcilia on Hierarchical and grouped me series 4 Quarterly data on visitor night from 1998:Q1 – 2013:Q4 From Na onal Visitor Survey, based on annual interviews of 120,000 Australians aged 15+, collected by Tourism Research Australia. Split by 7 states, 27 zones and 76 regions (a geographical hierarchy) Also split by purpose of travel Holiday Visi ng friends and rela ves (VFR) Business Other 304 bo om-level series

Slide 7

Slide 7 text

Spectacle sales Op mal Forecast Reconcilia on Hierarchical and grouped me series 5 Monthly UK sales data from 2000 – 2014 Provided by a large spectacle manufacturer Split by brand (26), gender (3), price range (6), materials (4), and stores (600) About 1 million bo om-level series

Slide 8

Slide 8 text

Spectacle sales Op mal Forecast Reconcilia on Hierarchical and grouped me series 5 Monthly UK sales data from 2000 – 2014 Provided by a large spectacle manufacturer Split by brand (26), gender (3), price range (6), materials (4), and stores (600) About 1 million bo om-level series

Slide 9

Slide 9 text

Spectacle sales Op mal Forecast Reconcilia on Hierarchical and grouped me series 5 Monthly UK sales data from 2000 – 2014 Provided by a large spectacle manufacturer Split by brand (26), gender (3), price range (6), materials (4), and stores (600) About 1 million bo om-level series

Slide 10

Slide 10 text

Spectacle sales Op mal Forecast Reconcilia on Hierarchical and grouped me series 5 Monthly UK sales data from 2000 – 2014 Provided by a large spectacle manufacturer Split by brand (26), gender (3), price range (6), materials (4), and stores (600) About 1 million bo om-level series

Slide 11

Slide 11 text

Hierarchical me series A hierarchical me series is a collec on of several me series that are linked together in a hierarchical structure. Total A AA AB AC B BA BB BC C CA CB CC Examples Labour turnover by occupa on Tourism by state and region Op mal Forecast Reconcilia on Hierarchical and grouped me series 6

Slide 12

Slide 12 text

Hierarchical me series A hierarchical me series is a collec on of several me series that are linked together in a hierarchical structure. Total A AA AB AC B BA BB BC C CA CB CC Examples Labour turnover by occupa on Tourism by state and region Op mal Forecast Reconcilia on Hierarchical and grouped me series 6

Slide 13

Slide 13 text

Hierarchical me series A hierarchical me series is a collec on of several me series that are linked together in a hierarchical structure. Total A AA AB AC B BA BB BC C CA CB CC Examples Labour turnover by occupa on Tourism by state and region Op mal Forecast Reconcilia on Hierarchical and grouped me series 6

Slide 14

Slide 14 text

Grouped me series A grouped me series is a collec on of me series that can be grouped together in a number of non-hierarchical ways. Total A AX AY B BX BY Total X AX BX Y AY BY Examples Labour turnover by occupa on and state Tourism by region and purpose of travel Spectacle sales by brand, gender, stores, etc. Op mal Forecast Reconcilia on Hierarchical and grouped me series 7

Slide 15

Slide 15 text

Grouped me series A grouped me series is a collec on of me series that can be grouped together in a number of non-hierarchical ways. Total A AX AY B BX BY Total X AX BX Y AY BY Examples Labour turnover by occupa on and state Tourism by region and purpose of travel Spectacle sales by brand, gender, stores, etc. Op mal Forecast Reconcilia on Hierarchical and grouped me series 7

Slide 16

Slide 16 text

Grouped me series A grouped me series is a collec on of me series that can be grouped together in a number of non-hierarchical ways. Total A AX AY B BX BY Total X AX BX Y AY BY Examples Labour turnover by occupa on and state Tourism by region and purpose of travel Spectacle sales by brand, gender, stores, etc. Op mal Forecast Reconcilia on Hierarchical and grouped me series 7

Slide 17

Slide 17 text

Grouped me series A grouped me series is a collec on of me series that can be grouped together in a number of non-hierarchical ways. Total A AX AY B BX BY Total X AX BX Y AY BY Examples Labour turnover by occupa on and state Tourism by region and purpose of travel Spectacle sales by brand, gender, stores, etc. Op mal Forecast Reconcilia on Hierarchical and grouped me series 7

Slide 18

Slide 18 text

tl;dr 1 Forecast all series at all levels of aggrega on using an automa c forecas ng algorithm (e.g., ets, auto.arima, ...) 2 Reconcile the resul ng forecasts so they add up correctly using least squares op miza on (i.e., find closest reconciled forecasts to the original forecasts). 3 This is all available in the hts package in R. Op mal Forecast Reconcilia on Hierarchical and grouped me series 8

Slide 19

Slide 19 text

tl;dr 1 Forecast all series at all levels of aggrega on using an automa c forecas ng algorithm (e.g., ets, auto.arima, ...) 2 Reconcile the resul ng forecasts so they add up correctly using least squares op miza on (i.e., find closest reconciled forecasts to the original forecasts). 3 This is all available in the hts package in R. Op mal Forecast Reconcilia on Hierarchical and grouped me series 8

Slide 20

Slide 20 text

tl;dr 1 Forecast all series at all levels of aggrega on using an automa c forecas ng algorithm (e.g., ets, auto.arima, ...) 2 Reconcile the resul ng forecasts so they add up correctly using least squares op miza on (i.e., find closest reconciled forecasts to the original forecasts). 3 This is all available in the hts package in R. Op mal Forecast Reconcilia on Hierarchical and grouped me series 8

Slide 21

Slide 21 text

Hierarchical me series Total A B C Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.

Slide 22

Slide 22 text

Hierarchical me series Total A B C Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.

Slide 23

Slide 23 text

Hierarchical me series Total A B C yt = [yt , yA,t , yB,t , yC,t ] =     1 1 1 1 0 0 0 1 0 0 0 1       yA,t yB,t yC,t   Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.

Slide 24

Slide 24 text

Hierarchical me series Total A B C yt = [yt , yA,t , yB,t , yC,t ] =     1 1 1 1 0 0 0 1 0 0 0 1     S   yA,t yB,t yC,t   Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.

Slide 25

Slide 25 text

Hierarchical me series Total A B C yt = [yt , yA,t , yB,t , yC,t ] =     1 1 1 1 0 0 0 1 0 0 0 1     S   yA,t yB,t yC,t   bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.

Slide 26

Slide 26 text

Hierarchical me series Total A B C yt = [yt , yA,t , yB,t , yC,t ] =     1 1 1 1 0 0 0 1 0 0 0 1     S   yA,t yB,t yC,t   bt yt = Sbt Op mal Forecast Reconcilia on Hierarchical and grouped me series 9 yt : observed aggregate of all series at me t. yX,t : observa on on series X at me t. bt : vector of all series at bo om level in me t.

Slide 27

Slide 27 text

Hierarchical me series Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yB,t yC,t yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 10

Slide 28

Slide 28 text

Hierarchical me series Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yB,t yC,t yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 10

Slide 29

Slide 29 text

Hierarchical me series Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yB,t yC,t yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 10 yt = Sbt

Slide 30

Slide 30 text

Grouped data AX AY A BX BY B X Y Total yt =             yt yA,t yB,t yX,t yY,t yAX,t yAY,t yBX,t yBY,t             =             1 1 1 1 1 1 0 0 0 0 1 1 1 0 1 0 0 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1             S    yAX,t yAY,t yBX,t yBY,t    bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 11

Slide 31

Slide 31 text

Grouped data AX AY A BX BY B X Y Total yt =             yt yA,t yB,t yX,t yY,t yAX,t yAY,t yBX,t yBY,t             =             1 1 1 1 1 1 0 0 0 0 1 1 1 0 1 0 0 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1             S    yAX,t yAY,t yBX,t yBY,t    bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 11

Slide 32

Slide 32 text

Grouped data AX AY A BX BY B X Y Total yt =             yt yA,t yB,t yX,t yY,t yAX,t yAY,t yBX,t yBY,t             =             1 1 1 1 1 1 0 0 0 0 1 1 1 0 1 0 0 1 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1             S    yAX,t yAY,t yBX,t yBY,t    bt Op mal Forecast Reconcilia on Hierarchical and grouped me series 11 yt = Sbt

Slide 33

Slide 33 text

Hierarchical and grouped me series Every collec on of me series with aggrega on constraints can be wri en as yt = Sbt where yt is a vector of all series at me t bt is a vector of the most disaggregated series at me t S is a “summing matrix” containing the aggrega on constraints. Op mal Forecast Reconcilia on Hierarchical and grouped me series 12

Slide 34

Slide 34 text

Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on Minimum trace reconcilia on 13

Slide 35

Slide 35 text

Forecas ng nota on Let ˆ yn (h) be vector of ini al h-step forecasts, made at me n, stacked in same order as yt. (In general, they will not “add up”.) Reconciled forecasts must be of the form: ˜ yn (h) = SPˆ yn (h) for some matrix P. P extracts and combines base forecasts ˆ yn (h) to get bo om-level forecasts. S adds them up Op mal Forecast Reconcilia on Minimum trace reconcilia on 14

Slide 36

Slide 36 text

Forecas ng nota on Let ˆ yn (h) be vector of ini al h-step forecasts, made at me n, stacked in same order as yt. (In general, they will not “add up”.) Reconciled forecasts must be of the form: ˜ yn (h) = SPˆ yn (h) for some matrix P. P extracts and combines base forecasts ˆ yn (h) to get bo om-level forecasts. S adds them up Op mal Forecast Reconcilia on Minimum trace reconcilia on 14

Slide 37

Slide 37 text

Forecas ng nota on Let ˆ yn (h) be vector of ini al h-step forecasts, made at me n, stacked in same order as yt. (In general, they will not “add up”.) Reconciled forecasts must be of the form: ˜ yn (h) = SPˆ yn (h) for some matrix P. P extracts and combines base forecasts ˆ yn (h) to get bo om-level forecasts. S adds them up Op mal Forecast Reconcilia on Minimum trace reconcilia on 14

Slide 38

Slide 38 text

Forecas ng nota on Let ˆ yn (h) be vector of ini al h-step forecasts, made at me n, stacked in same order as yt. (In general, they will not “add up”.) Reconciled forecasts must be of the form: ˜ yn (h) = SPˆ yn (h) for some matrix P. P extracts and combines base forecasts ˆ yn (h) to get bo om-level forecasts. S adds them up Op mal Forecast Reconcilia on Minimum trace reconcilia on 14

Slide 39

Slide 39 text

Forecas ng nota on Let ˆ yn (h) be vector of ini al h-step forecasts, made at me n, stacked in same order as yt. (In general, they will not “add up”.) Reconciled forecasts must be of the form: ˜ yn (h) = SPˆ yn (h) for some matrix P. P extracts and combines base forecasts ˆ yn (h) to get bo om-level forecasts. S adds them up Op mal Forecast Reconcilia on Minimum trace reconcilia on 14

Slide 40

Slide 40 text

General proper es: bias and variance ˜ yn (h) = SPˆ yn (h) Bias Reconciled forecasts are unbiased iff SPS = S. Variance Let error variance of h-step base forecasts ˆ yn (h) be Wh = Var[yn+h − ˆ yn (h) | y1 , . . . , yn ] Then the error variance of the corresponding reconciled forecasts is Var[yn+h − ˜ yn (h) | y1 , . . . , yn ] = SPWh P S Op mal Forecast Reconcilia on Minimum trace reconcilia on 15

Slide 41

Slide 41 text

General proper es: bias and variance ˜ yn (h) = SPˆ yn (h) Bias Reconciled forecasts are unbiased iff SPS = S. Variance Let error variance of h-step base forecasts ˆ yn (h) be Wh = Var[yn+h − ˆ yn (h) | y1 , . . . , yn ] Then the error variance of the corresponding reconciled forecasts is Var[yn+h − ˜ yn (h) | y1 , . . . , yn ] = SPWh P S Op mal Forecast Reconcilia on Minimum trace reconcilia on 15

Slide 42

Slide 42 text

General proper es: bias and variance ˜ yn (h) = SPˆ yn (h) Bias Reconciled forecasts are unbiased iff SPS = S. Variance Let error variance of h-step base forecasts ˆ yn (h) be Wh = Var[yn+h − ˆ yn (h) | y1 , . . . , yn ] Then the error variance of the corresponding reconciled forecasts is Var[yn+h − ˜ yn (h) | y1 , . . . , yn ] = SPWh P S Op mal Forecast Reconcilia on Minimum trace reconcilia on 15

Slide 43

Slide 43 text

Op mal forecast reconcilia on ˜ yn (h) = SPˆ yn (h) Theorem: MinT Reconcilia on If P sa sfies SPS = S, then minP = trace[SPWh P S ] has solu on P = (S W−1 h S)−1S W−1 h . Reconciled forecasts Base forecasts Assume that Wh = kh W1 to simplify computa ons. Op mal Forecast Reconcilia on Minimum trace reconcilia on 16 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 44

Slide 44 text

Op mal forecast reconcilia on ˜ yn (h) = SPˆ yn (h) Theorem: MinT Reconcilia on If P sa sfies SPS = S, then minP = trace[SPWh P S ] has solu on P = (S W−1 h S)−1S W−1 h . Reconciled forecasts Base forecasts Assume that Wh = kh W1 to simplify computa ons. Op mal Forecast Reconcilia on Minimum trace reconcilia on 16 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 45

Slide 45 text

Op mal forecast reconcilia on ˜ yn (h) = SPˆ yn (h) Theorem: MinT Reconcilia on If P sa sfies SPS = S, then minP = trace[SPWh P S ] has solu on P = (S W−1 h S)−1S W−1 h . Reconciled forecasts Base forecasts Assume that Wh = kh W1 to simplify computa ons. Op mal Forecast Reconcilia on Minimum trace reconcilia on 16 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 46

Slide 46 text

Op mal forecast reconcilia on ˜ yn (h) = SPˆ yn (h) Theorem: MinT Reconcilia on If P sa sfies SPS = S, then minP = trace[SPWh P S ] has solu on P = (S W−1 h S)−1S W−1 h . Reconciled forecasts Base forecasts Assume that Wh = kh W1 to simplify computa ons. Op mal Forecast Reconcilia on Minimum trace reconcilia on 16 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 47

Slide 47 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)

Slide 48

Slide 48 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)

Slide 49

Slide 49 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)

Slide 50

Slide 50 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)

Slide 51

Slide 51 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)

Slide 52

Slide 52 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 1: OLS Assume W1 ≈ kI. ˜ yn (h) = S(S S)−1S ˆ yn (h) Reconcilia on does not depend on data Works surprisingly well. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 17 ˜ yn (h) = S(S W−1 1 S)−1S W−1 1 ˆ yn (h)

Slide 53

Slide 53 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 2: WLS Approximate W1 by its diagonal. Easy to es mate, and places weight where we have best forecasts. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 18 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 54

Slide 54 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 2: WLS Approximate W1 by its diagonal. Easy to es mate, and places weight where we have best forecasts. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 18 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 55

Slide 55 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 2: WLS Approximate W1 by its diagonal. Easy to es mate, and places weight where we have best forecasts. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 18 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 56

Slide 56 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 2: WLS Approximate W1 by its diagonal. Easy to es mate, and places weight where we have best forecasts. S ll need to es mate covariance matrix to produce predic on intervals. Op mal Forecast Reconcilia on Minimum trace reconcilia on 18 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 57

Slide 57 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 3: GLS Es mate W1 using shrinkage to the diagonal. Allows for covariances. Difficult to compute for large numbers of me series. Op mal Forecast Reconcilia on Minimum trace reconcilia on 19 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 58

Slide 58 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 3: GLS Es mate W1 using shrinkage to the diagonal. Allows for covariances. Difficult to compute for large numbers of me series. Op mal Forecast Reconcilia on Minimum trace reconcilia on 19 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 59

Slide 59 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 3: GLS Es mate W1 using shrinkage to the diagonal. Allows for covariances. Difficult to compute for large numbers of me series. Op mal Forecast Reconcilia on Minimum trace reconcilia on 19 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 60

Slide 60 text

Op mal forecast reconcilia on Reconciled forecasts Base forecasts Solu on 3: GLS Es mate W1 using shrinkage to the diagonal. Allows for covariances. Difficult to compute for large numbers of me series. Op mal Forecast Reconcilia on Minimum trace reconcilia on 19 ˜ yn (h) = S(S W−1 h S)−1S W−1 h ˆ yn (h)

Slide 61

Slide 61 text

Australian tourism Op mal Forecast Reconcilia on Minimum trace reconcilia on 20

Slide 62

Slide 62 text

Australian tourism Op mal Forecast Reconcilia on Minimum trace reconcilia on 20 Hierarchy: States (7) Zones (27) Regions (82)

Slide 63

Slide 63 text

Australian tourism Op mal Forecast Reconcilia on Minimum trace reconcilia on 20 Hierarchy: States (7) Zones (27) Regions (82) Base forecasts ETS (exponen al smoothing) models

Slide 64

Slide 64 text

Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: Total Year Visitor nights 1998 2000 2002 2004 2006 2008 60000 65000 70000 75000 80000 85000

Slide 65

Slide 65 text

Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: NSW Year Visitor nights 1998 2000 2002 2004 2006 2008 18000 22000 26000 30000

Slide 66

Slide 66 text

Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: VIC Year Visitor nights 1998 2000 2002 2004 2006 2008 10000 12000 14000 16000 18000

Slide 67

Slide 67 text

Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: Nth.Coast.NSW Year Visitor nights 1998 2000 2002 2004 2006 2008 5000 6000 7000 8000 9000

Slide 68

Slide 68 text

Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: Metro.QLD Year Visitor nights 1998 2000 2002 2004 2006 2008 8000 9000 11000 13000

Slide 69

Slide 69 text

Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: Sth.WA Year Visitor nights 1998 2000 2002 2004 2006 2008 400 600 800 1000 1200 1400

Slide 70

Slide 70 text

Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: X201.Melbourne Year Visitor nights 1998 2000 2002 2004 2006 2008 4000 4500 5000 5500 6000

Slide 71

Slide 71 text

Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: X402.Murraylands Year Visitor nights 1998 2000 2002 2004 2006 2008 0 100 200 300

Slide 72

Slide 72 text

Base forecasts Op mal Forecast Reconcilia on Minimum trace reconcilia on 21 Domestic tourism forecasts: X809.Daly Year Visitor nights 1998 2000 2002 2004 2006 2008 0 20 40 60 80 100

Slide 73

Slide 73 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 74

Slide 74 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 75

Slide 75 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 76

Slide 76 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 77

Slide 77 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 78

Slide 78 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 79

Slide 79 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 80

Slide 80 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 81

Slide 81 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 82

Slide 82 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 83

Slide 83 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 84

Slide 84 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 85

Slide 85 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 86

Slide 86 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 87

Slide 87 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 88

Slide 88 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 89

Slide 89 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 90

Slide 90 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 91

Slide 91 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 92

Slide 92 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 1 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 93

Slide 93 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 2 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 94

Slide 94 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 3 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 95

Slide 95 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 4 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 96

Slide 96 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 5 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 97

Slide 97 text

Forecast evalua on Op mal Forecast Reconcilia on Minimum trace reconcilia on 22 Training sets Test sets h = 6 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q time

Slide 98

Slide 98 text

Hierarchy: states, zones, regions Forecast horizon RMSE h = 1 h = 2 h = 3 h = 4 h = 5 h = 6 Ave Australia Base 1762.04 1770.29 1766.02 1818.82 1705.35 1721.17 1757.28 Bo om 1736.92 1742.69 1722.79 1752.74 1666.73 1687.43 1718.22 OLS 1747.60 1757.68 1751.77 1800.67 1686.00 1706.45 1741.69 WLS 1705.21 1715.87 1703.75 1729.56 1627.79 1661.24 1690.57 GLS 1704.64 1715.60 1705.31 1729.04 1626.36 1661.64 1690.43 States Base 399.77 404.16 401.92 407.26 395.38 401.17 401.61 Bo om 404.29 406.95 404.96 409.02 399.80 401.55 404.43 OLS 404.47 407.62 405.43 413.79 401.10 404.90 406.22 WLS 398.84 402.12 400.71 405.03 394.76 398.23 399.95 GLS 398.84 402.16 400.86 405.03 394.59 398.22 399.95 Regions Base 93.15 93.38 93.45 93.79 93.50 93.56 93.47 Bo om 93.15 93.38 93.45 93.79 93.50 93.56 93.47 OLS 93.28 93.53 93.64 94.17 93.78 93.88 93.71 WLS 93.02 93.32 93.38 93.72 93.39 93.53 93.39 GLS 92.98 93.27 93.34 93.66 93.34 93.46 93.34 Op mal Forecast Reconcilia on Minimum trace reconcilia on 23

Slide 99

Slide 99 text

Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on Fast computa onal tricks 24

Slide 100

Slide 100 text

Fast computa on: hierarchical data Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yB,t yC,t yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Fast computa onal tricks 25 yt = Sbt

Slide 101

Slide 101 text

Fast computa on: hierarchical data Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yAX,t yAY,t yAZ,t yB,t yBX,t yBY,t yBZ,t yC,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Fast computa onal tricks 26 yt = Sbt

Slide 102

Slide 102 text

Fast computa on: hierarchical data Total A AX AY AZ B BX BY BZ C CX CY CZ yt =             yt yA,t yAX,t yAY,t yAZ,t yB,t yBX,t yBY,t yBZ,t yC,t yCX,t yCY,t yCZ,t             =             1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1             S        yAX,t yAY,t yAZ,t yBX,t yBY,t yBZ,t yCX,t yCY,t yCZ,t        bt Op mal Forecast Reconcilia on Fast computa onal tricks 26 yt = Sbt

Slide 103

Slide 103 text

Fast computa on: hierarchies Think of the hierarchy as a tree of trees: Total T1 T2 ... TK Then the summing matrix contains k smaller summing matrices: S =       1n1 1n2 · · · 1nK S1 0 · · · 0 0 S2 · · · 0 . . . . . . ... . . . 0 0 · · · SK       where 1n is an n-vector of ones and tree Ti has ni terminal nodes. Op mal Forecast Reconcilia on Fast computa onal tricks 27

Slide 104

Slide 104 text

Fast computa on: hierarchies Think of the hierarchy as a tree of trees: Total T1 T2 ... TK Then the summing matrix contains k smaller summing matrices: S =       1n1 1n2 · · · 1nK S1 0 · · · 0 0 S2 · · · 0 . . . . . . ... . . . 0 0 · · · SK       where 1n is an n-vector of ones and tree Ti has ni terminal nodes. Op mal Forecast Reconcilia on Fast computa onal tricks 27

Slide 105

Slide 105 text

Fast computa on: hierarchies SΛS =     S1 Λ1 S1 0 · · · 0 0 S2 Λ2 S2 · · · 0 . . . . . . ... . . . 0 0 · · · SK ΛK SK     + λ0 Jn λ0 is the top le element of Λ; Λk is a block of Λ, corresponding to tree Tk; Jn is a matrix of ones; n = k nk. Now apply the Sherman-Morrison formula ... Op mal Forecast Reconcilia on Fast computa onal tricks 28

Slide 106

Slide 106 text

Fast computa on: hierarchies SΛS =     S1 Λ1 S1 0 · · · 0 0 S2 Λ2 S2 · · · 0 . . . . . . ... . . . 0 0 · · · SK ΛK SK     + λ0 Jn λ0 is the top le element of Λ; Λk is a block of Λ, corresponding to tree Tk; Jn is a matrix of ones; n = k nk. Now apply the Sherman-Morrison formula ... Op mal Forecast Reconcilia on Fast computa onal tricks 28

Slide 107

Slide 107 text

Fast computa on: hierarchies (S ΛS)−1 =     (S1 Λ1 S1 )−1 0 · · · 0 0 (S2 Λ2 S2 )−1 · · · 0 . . . . . . ... . . . 0 0 · · · (SK ΛK SK )−1     − cS0 S0 can be par oned into K2 blocks, with the (k, ) block (of dimension nk × n ) being (Sk Λk Sk )−1Jnk ,n (S Λ S )−1 Jnk ,n is a nk × n matrix of ones. c−1 = λ−1 0 + k 1nk (Sk Λk Sk )−11nk . Each Sk Λk Sk can be inverted similarly. S Λy can also be computed recursively. Op mal Forecast Reconcilia on Fast computa onal tricks 29

Slide 108

Slide 108 text

Fast computa on: hierarchies (S ΛS)−1 =     (S1 Λ1 S1 )−1 0 · · · 0 0 (S2 Λ2 S2 )−1 · · · 0 . . . . . . ... . . . 0 0 · · · (SK ΛK SK )−1     − cS0 S0 can be par oned into K2 blocks, with the (k, ) block (of dimension nk × n ) being (Sk Λk Sk )−1Jnk ,n (S Λ S )−1 Jnk ,n is a nk × n matrix of ones. c−1 = λ−1 0 + k 1nk (Sk Λk Sk )−11nk . Each Sk Λk Sk can be inverted similarly. S Λy can also be computed recursively. Op mal Forecast Reconcilia on Fast computa onal tricks 29 The recursive calcula ons can be done in such a way that we never store any of the large matrices involved.

Slide 109

Slide 109 text

Fast computa on A similar algorithm has been developed for grouped me series with two groups. When the me series are not strictly hierarchical and have more than two grouping variables: Use sparse matrix storage and arithme c. Use itera ve approxima on for inver ng large sparse matrices. Paige & Saunders (1982) ACM Trans. Math. So ware Op mal Forecast Reconcilia on Fast computa onal tricks 30

Slide 110

Slide 110 text

Fast computa on A similar algorithm has been developed for grouped me series with two groups. When the me series are not strictly hierarchical and have more than two grouping variables: Use sparse matrix storage and arithme c. Use itera ve approxima on for inver ng large sparse matrices. Paige & Saunders (1982) ACM Trans. Math. So ware Op mal Forecast Reconcilia on Fast computa onal tricks 30

Slide 111

Slide 111 text

Fast computa on A similar algorithm has been developed for grouped me series with two groups. When the me series are not strictly hierarchical and have more than two grouping variables: Use sparse matrix storage and arithme c. Use itera ve approxima on for inver ng large sparse matrices. Paige & Saunders (1982) ACM Trans. Math. So ware Op mal Forecast Reconcilia on Fast computa onal tricks 30

Slide 112

Slide 112 text

Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on Temporal hierarchies 31

Slide 113

Slide 113 text

Temporal hierarchies Annual Semi-Annual1 Q1 Q2 Semi-Annual2 Q3 Q4 Basic idea: ¯ Forecast series at each available frequency. ¯ Op mally reconcile forecasts within the same year. Op mal Forecast Reconcilia on Temporal hierarchies 32

Slide 114

Slide 114 text

Temporal hierarchies Annual Semi-Annual1 Q1 Q2 Semi-Annual2 Q3 Q4 Basic idea: ¯ Forecast series at each available frequency. ¯ Op mally reconcile forecasts within the same year. Op mal Forecast Reconcilia on Temporal hierarchies 32

Slide 115

Slide 115 text

Monthly series Annual Semi-Annual1 Q1 M1 M2 M3 Q2 M4 M5 M6 Semi-Annual2 Q3 M7 M8 M9 Q4 M10 M11 M12 k = 2, 4, 12 nodes k = 3, 6, 12 nodes Why not k = 2, 3, 4, 6, 12 nodes? Op mal Forecast Reconcilia on Temporal hierarchies 33

Slide 116

Slide 116 text

Monthly series Annual FourM1 BiM1 M1 M2 BiM2 M3 M4 FourM2 BiM3 M5 M6 BiM4 M7 M8 FourM3 BiM5 M9 M10 BiM6 M11 M12 k = 2, 4, 12 nodes k = 3, 6, 12 nodes Why not k = 2, 3, 4, 6, 12 nodes? Op mal Forecast Reconcilia on Temporal hierarchies 33

Slide 117

Slide 117 text

Monthly series Annual FourM1 BiM1 M1 M2 BiM2 M3 M4 FourM2 BiM3 M5 M6 BiM4 M7 M8 FourM3 BiM5 M9 M10 BiM6 M11 M12 k = 2, 4, 12 nodes k = 3, 6, 12 nodes Why not k = 2, 3, 4, 6, 12 nodes? Op mal Forecast Reconcilia on Temporal hierarchies 33

Slide 118

Slide 118 text

Monthly data                   A SemiA1 SemiA2 FourM1 FourM2 FourM3 Q1 . . . Q4 BiM1 . . . BiM6 M1 . . . M12                   (28×1) =                   1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 . . . 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 . . . 0 0 0 0 0 0 0 0 0 0 1 1 I12                   S                M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12                Bt Op mal Forecast Reconcilia on Temporal hierarchies 34

Slide 119

Slide 119 text

In general For a me series y1 , . . . , yT, observed at frequency m, we generate aggregate series y[k] j = jk t=1+(j−1)k yt , for j = 1, . . . , T/k k ∈ F(m) = {factors of m}. A single unique hierarchy is only possible when there are no coprime pairs in F(m). Mk = m/k is seasonal period of aggregated series. Op mal Forecast Reconcilia on Temporal hierarchies 35

Slide 120

Slide 120 text

In general For a me series y1 , . . . , yT, observed at frequency m, we generate aggregate series y[k] j = jk t=1+(j−1)k yt , for j = 1, . . . , T/k k ∈ F(m) = {factors of m}. A single unique hierarchy is only possible when there are no coprime pairs in F(m). Mk = m/k is seasonal period of aggregated series. Op mal Forecast Reconcilia on Temporal hierarchies 35

Slide 121

Slide 121 text

In general For a me series y1 , . . . , yT, observed at frequency m, we generate aggregate series y[k] j = jk t=1+(j−1)k yt , for j = 1, . . . , T/k k ∈ F(m) = {factors of m}. A single unique hierarchy is only possible when there are no coprime pairs in F(m). Mk = m/k is seasonal period of aggregated series. Op mal Forecast Reconcilia on Temporal hierarchies 35

Slide 122

Slide 122 text

WLS weights Hierarchy variance scaling ΛH: diagonal. Series variance scaling ΛV: elements equal within aggrega on level. Structural scaling ΛS = diag(S1): elements equal to # nodes at each level. Depends only on seasonal period m. Independent of data and model. Allows forecasts where no errors available. Quarterly example ΛH = diag ˆ σ2 A , ˆ σ2 S1 , ˆ σ2 S2 , ˆ σ2 Q1 , ˆ σ2 Q2 , ˆ σ2 Q3 , ˆ σ2 Q4 ΛV = diag ˆ σ2 A , ˆ σ2 S , ˆ σ2 S , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q ΛS = diag 4, 2, 2, 1, 1, 1, 1 Op mal Forecast Reconcilia on Temporal hierarchies 36

Slide 123

Slide 123 text

WLS weights Hierarchy variance scaling ΛH: diagonal. Series variance scaling ΛV: elements equal within aggrega on level. Structural scaling ΛS = diag(S1): elements equal to # nodes at each level. Depends only on seasonal period m. Independent of data and model. Allows forecasts where no errors available. Quarterly example ΛH = diag ˆ σ2 A , ˆ σ2 S1 , ˆ σ2 S2 , ˆ σ2 Q1 , ˆ σ2 Q2 , ˆ σ2 Q3 , ˆ σ2 Q4 ΛV = diag ˆ σ2 A , ˆ σ2 S , ˆ σ2 S , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q ΛS = diag 4, 2, 2, 1, 1, 1, 1 Op mal Forecast Reconcilia on Temporal hierarchies 36

Slide 124

Slide 124 text

WLS weights Hierarchy variance scaling ΛH: diagonal. Series variance scaling ΛV: elements equal within aggrega on level. Structural scaling ΛS = diag(S1): elements equal to # nodes at each level. Depends only on seasonal period m. Independent of data and model. Allows forecasts where no errors available. Quarterly example ΛH = diag ˆ σ2 A , ˆ σ2 S1 , ˆ σ2 S2 , ˆ σ2 Q1 , ˆ σ2 Q2 , ˆ σ2 Q3 , ˆ σ2 Q4 ΛV = diag ˆ σ2 A , ˆ σ2 S , ˆ σ2 S , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q , ˆ σ2 Q ΛS = diag 4, 2, 2, 1, 1, 1, 1 Op mal Forecast Reconcilia on Temporal hierarchies 36

Slide 125

Slide 125 text

UK Accidents and Emergency Demand Op mal Forecast Reconcilia on Temporal hierarchies 37 1 2 3 4 5 6 5100 5300 5500 Annual (k=52) Forecast 2 4 6 8 10 12 2500 2600 2700 2800 2900 Semi−annual (k=26) Forecast 5 10 15 20 25 1250 1350 1450 Quarterly (k=13) Forecast 20 40 60 80 360 380 400 420 440 460 Monthly (k=4) Forecast 50 100 150 180 190 200 210 220 230 Bi−weekly (k=2) Forecast 50 100 150 200 250 300 90 95 100 105 110 Weekly (k=1) Forecast – – – – base reconciled

Slide 126

Slide 126 text

UK Accidents and Emergency Demand 1 Type 1 Departments — Major A&E 2 Type 2 Departments — Single Specialty 3 Type 3 Departments — Other A&E/Minor Injury 4 Total A endances 5 Type 1 Departments — Major A&E > 4 hrs 6 Type 2 Departments — Single Specialty > 4 hrs 7 Type 3 Departments — Other A&E/Minor Injury > 4 hrs 8 Total A endances > 4 hrs 9 Emergency Admissions via Type 1 A&E 10 Total Emergency Admissions via A&E 11 Other Emergency Admissions (i.e., not via A&E) 12 Total Emergency Admissions 13 Number of pa ents spending > 4 hrs from decision to admission Op mal Forecast Reconcilia on Temporal hierarchies 38

Slide 127

Slide 127 text

UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39

Slide 128

Slide 128 text

UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39

Slide 129

Slide 129 text

UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39

Slide 130

Slide 130 text

UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39

Slide 131

Slide 131 text

UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39

Slide 132

Slide 132 text

UK Accidents and Emergency Demand Minimum training set: all data except the last year Base forecasts using auto.arima(). Reconciled using WLSV. Mean Absolute Scaled Errors for 1, 4 and 13 weeks ahead using a rolling origin. Aggr. Level h Base Reconciled Change Weekly 1 1.6 1.3 −17.2% Weekly 4 1.9 1.5 −18.6% Weekly 13 2.3 1.9 −16.2% Weekly 1–52 2.0 1.9 −5.0% Annual 1 3.4 1.9 −42.9% Op mal Forecast Reconcilia on Temporal hierarchies 39

Slide 133

Slide 133 text

Experimental setup: M3 forecas ng compe on (Makridakis and Hibon, 2000, IJF). In total 3003 series. 1,428 monthly series with a test sample of 12 observa ons each. 756 quarterly series with a test sample of 8 observa ons each. Forecast each series with ETS models. Op mal Forecast Reconcilia on Temporal hierarchies 40

Slide 134

Slide 134 text

Experimental setup: M3 forecas ng compe on (Makridakis and Hibon, 2000, IJF). In total 3003 series. 1,428 monthly series with a test sample of 12 observa ons each. 756 quarterly series with a test sample of 8 observa ons each. Forecast each series with ETS models. Op mal Forecast Reconcilia on Temporal hierarchies 40

Slide 135

Slide 135 text

Experimental setup: M3 forecas ng compe on (Makridakis and Hibon, 2000, IJF). In total 3003 series. 1,428 monthly series with a test sample of 12 observa ons each. 756 quarterly series with a test sample of 8 observa ons each. Forecast each series with ETS models. Op mal Forecast Reconcilia on Temporal hierarchies 40

Slide 136

Slide 136 text

Experimental setup: M3 forecas ng compe on (Makridakis and Hibon, 2000, IJF). In total 3003 series. 1,428 monthly series with a test sample of 12 observa ons each. 756 quarterly series with a test sample of 8 observa ons each. Forecast each series with ETS models. Op mal Forecast Reconcilia on Temporal hierarchies 40

Slide 137

Slide 137 text

Results: Monthly MAE percent difference rela ve to base max h BU WLSH WLSV WLSS Annual 1 −19.6 −22.0 −22.0 −25.1 Semi-annual 3 0.6 −4.0 −3.6 −5.4 Four-monthly 4 2.0 −2.4 −2.2 −3.0 Quarterly 6 2.4 −1.6 −1.7 −2.8 Bi-monthly 9 0.7 −2.9 −3.3 −4.3 Monthly 18 0.0 −2.2 −3.2 −3.9 Op mal Forecast Reconcilia on Temporal hierarchies 41

Slide 138

Slide 138 text

Results: Quarterly MAE percent difference rela ve to base max h BU WLSH WLSV WLSS Annual 1 −20.9 -22.7 −22.8 -22.7 Semi-annual 3 −4.5 −6.0 −6.2 -4.8 Quarterly 6 0.0 −0.2 −1.1 -0.3 Op mal Forecast Reconcilia on Temporal hierarchies 42

Slide 139

Slide 139 text

Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on Probabilis c reconcilia on 43

Slide 140

Slide 140 text

Coherent density forecasts Defini on: Coherence Suppose yt ∈ Rn. yt is coherent if yt lies in an m-dimensional subspace of Rn spanned by the columns of the summing matrix S. Defini on: Coherent density forecasts Any density p(yt+h ) such that p(yt+h ) = 0 for all yt+h in the null space of S. Coherent point forecasts: ˜ yT+h|T = SPˆ yT+h. Coherent variance forecasts (assuming unbiasedness): ˜ ΣT+h = SPˆ ΣT+h P S Op mal Forecast Reconcilia on Probabilis c reconcilia on 44

Slide 141

Slide 141 text

Coherent density forecasts Defini on: Coherence Suppose yt ∈ Rn. yt is coherent if yt lies in an m-dimensional subspace of Rn spanned by the columns of the summing matrix S. Defini on: Coherent density forecasts Any density p(yt+h ) such that p(yt+h ) = 0 for all yt+h in the null space of S. Coherent point forecasts: ˜ yT+h|T = SPˆ yT+h. Coherent variance forecasts (assuming unbiasedness): ˜ ΣT+h = SPˆ ΣT+h P S Op mal Forecast Reconcilia on Probabilis c reconcilia on 44

Slide 142

Slide 142 text

Coherent Gaussian forecasts yT+h|T ∼ N(˜ yT+h|T , ˜ ΣT+h ) Let L be the Energy Score (a proper scoring rule): L(˜ YT+h , yT+h ) = E ˜ YT+h − yT+h α − 1 2 E ˜ YT+h − ˜ YT+h α for α ∈ (0, 2], where ˜ YT+h and ˜ YT+h are independent rvs from N(˜ yT+h|T , ˜ ΣT+h ). There is no closed form expression for L(˜ YT+h , yT+h ) for α ∈ (0, 2) under the Gaussian predic ve distribu on. When α = 2, L(˜ YT+h , yT+h ) = ˜ YT+h − yT+h 2 This is equivalent to MinT solu on. Op mal Forecast Reconcilia on Probabilis c reconcilia on 45

Slide 143

Slide 143 text

Coherent nonparametric forecasts 1 Simulate forecast distribu ons at each bo om level node using univariate models 2 Compute empirical copulas for each parent+children group. Equivalent to “bo om-up” point forecas ng. No reconcilia on involved. Successfully applied to forecas ng smart-metre electricity demand (Ben Taieb, Taylor, Hyndman, 2017) Op mal Forecast Reconcilia on Probabilis c reconcilia on 46

Slide 144

Slide 144 text

Coherent nonparametric forecasts 1 Simulate forecast distribu ons at each bo om level node using univariate models 2 Compute empirical copulas for each parent+children group. Equivalent to “bo om-up” point forecas ng. No reconcilia on involved. Successfully applied to forecas ng smart-metre electricity demand (Ben Taieb, Taylor, Hyndman, 2017) Op mal Forecast Reconcilia on Probabilis c reconcilia on 46

Slide 145

Slide 145 text

Outline 1 Hierarchical and grouped me series 2 Minimum trace reconcilia on 3 Fast computa onal tricks 4 Temporal hierarchies 5 Probabilis c reconcilia on 6 References Op mal Forecast Reconcilia on References 47

Slide 146

Slide 146 text

References RJ Hyndman, RA Ahmed, G Athanasopoulos, and HL Shang (2011). Op mal combina on forecasts for hierarchical me series. Computa onal Sta s cs & Data Analysis 55(9), 2579–2589. RJ Hyndman, A Lee, and E Wang (2016). Fast computa on of reconciled forecasts for hierarchical and grouped me series. Computa onal Sta s cs & Data Analysis 97, 16–32 SL Wickramasuriya, G Athanasopoulos, and RJ Hyndman (2015). Forecas ng hierarchical and grouped me series through trace minimiza on. Working paper. Dept Econometrics & Business Sta s cs, Monash University G Athanasopoulos, RJ Hyndman, N Kourentzes, and F Petropoulos (2017). Forecas ng with temporal hierarchies. European Journal of Opera onal Research 262(1), 60–74 S Ben Taieb, JW Taylor, and RJ Hyndman (2017). Hierarchical Probabilis c Forecas ng of Electricity Demand with Smart Meter Data. Working paper. Dept Econometrics & Business Sta s cs, Monash University RJ Hyndman, A Lee, E Wang, and S Wickramasuriya (2017). hts: Hierarchical and Grouped Time Series. Version 5.1.4. https://CRAN.R-project.org/package=hts RJ Hyndman and N Kourentzes (2016). thief: Temporal Hierarchical Forecas ng. Version 0.2. https://CRAN.R-project.org/package=thief Op mal Forecast Reconcilia on References 48

Slide 147

Slide 147 text

References RJ Hyndman, RA Ahmed, G Athanasopoulos, and HL Shang (2011). Op mal combina on forecasts for hierarchical me series. Computa onal Sta s cs & Data Analysis 55(9), 2579–2589. RJ Hyndman, A Lee, and E Wang (2016). Fast computa on of reconciled forecasts for hierarchical and grouped me series. Computa onal Sta s cs & Data Analysis 97, 16–32 SL Wickramasuriya, G Athanasopoulos, and RJ Hyndman (2015). Forecas ng hierarchical and grouped me series through trace minimiza on. Working paper. Dept Econometrics & Business Sta s cs, Monash University G Athanasopoulos, RJ Hyndman, N Kourentzes, and F Petropoulos (2017). Forecas ng with temporal hierarchies. European Journal of Opera onal Research 262(1), 60–74 S Ben Taieb, JW Taylor, and RJ Hyndman (2017). Hierarchical Probabilis c Forecas ng of Electricity Demand with Smart Meter Data. Working paper. Dept Econometrics & Business Sta s cs, Monash University RJ Hyndman, A Lee, E Wang, and S Wickramasuriya (2017). hts: Hierarchical and Grouped Time Series. Version 5.1.4. https://CRAN.R-project.org/package=hts RJ Hyndman and N Kourentzes (2016). thief: Temporal Hierarchical Forecas ng. Version 0.2. https://CRAN.R-project.org/package=thief Op mal Forecast Reconcilia on References 48 ¯ More informa on: robjhyndman.com