to accelerate the renewable energy transition Inge van den Ende Data Scientist www.dexterenergy.ai Photo by Nicholas Doherty on Unsplash Eindhoven 30 November 2023
interval Training Calibration Prediction 2. Predict on calibration set and get conformity scores y Calibration set Point forecast Actual value Create a sorted list with absolute conformity scores Conformity scores Number of scores Select the quantile of the prediction interval quantile Conformity scores Number of scores
interval Training Calibration Prediction Create a sorted list with absolute conformity scores |e| 1 |e| 2 … |e| N-1 |e| N |e| 1 |e| 2 … |e| N-1 |e| N Select the quantile of the prediction interval index = ((1 - ⍺) * n) - 1 e 2 2. Predict on calibration set and get conformity scores y Calibration set Point forecast Actual value e 3 e 1
Prediction 3. Predict with point forecast model y Prediction set Point forecast y Prediction set Point forecast = y Point forecast + selected conformity score = y + |e| 2 Add prediction interval based on conformity scores Point forecast - selected conformity score = y - |e| 2
Training Calibration Prediction 1. Train point forecast model ŷ = f ( X ) 2. Predict on calibration set and get conformity scores 3. Predict with point forecast model & add prediction interval based on conformity scores y Calibration set Point forecast Actual value y Prediction set Point forecast Prediction interval
some disadvantages Model agnostic: Any model can be used Disadvantages Statistical guarantee: valid coverage No distribution assumption needed Constant over the prediction set A single prediction interval provides less information then a distribution
some disadvantages Model agnostic: Any model can be used Disadvantages Statistical guarantee: valid coverage No distribution assumption needed Constant over the prediction set A single prediction interval provides less information then a distribution Solution will be given in next slides
Calibration Prediction 1. Train a probabilistic forecast model on the training set [ŷ q01 …ŷ q01 ]= f ( X ) t -1 Time y Actuals Forecast Point forecast q80 q60 q40 q20 y Probability density slice For example: conformalized quantile regression
Quantile regression Conformal prediction Asymptotically consistent Statistical guarantee of valid coverage Takes into account local variability of the input space Basic application does not adapt to input space Conformalized quantile regression Statistical guarantee of valid coverage Takes into account local variability of the input space 26
factor Training Calibration Prediction Per interval of the probabilistic prediction 2. Predict on calibration set and get conformity scores y Calibration set Mean forecast Actual value 70% forecast 30% forecast 27 Create a sorted list with the conformity scores = residuals e 1 e 2 … e N-1 e N Select the quantile of the prediction interval index = ((1 - ⍺) * n) - 1 Correction can be positive or negative: Positive = wider distribution Negative = more narrow e 1 e 2 … e N-1 e N
with probabilistic forecast model Calibrate distribution based on conformity scores Calibrated probabilistic forecast Probabilistic forecast Per interval of the probabilistic prediction
the forecasted distribution Training Calibration Prediction 1. Train probabilistic forecast model [ŷ q01 …ŷ q01 ]= f ( X ) 2. Predict on calibration set and get conformity scores for every quantile 3. Predict on test set and calibrate that distribution y Calibration set y Prediction set
can be used Advantages Statistical guarantee: valid coverage No distribution assumption needed Varies over the prediction set A distribution provides more information then a single prediction interval Assumption: exchangeability Disadvantage