Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lab 29: Shiny Crude Oil Forecast (Multivariate ARIMA) App

71262b1f27ccad7e2bf0c8a2ca518b93?s=47 Matt Dancho
February 19, 2020

Lab 29: Shiny Crude Oil Forecast (Multivariate ARIMA) App

Time Series Fans - This one is for you! In Lab 29, you learn a workflow for Multivariate ARIMA Forecasting using Lagged Predictors, one of the most critical techniques in time series forecasting.

You learn to forecast using the new Fable library, part of the tidyverts ecosystem of time series & forecasting tools. We connect to the Quandl API to collect Energy Data & package the analysis in a Shiny Web Application for automated forecasting.

71262b1f27ccad7e2bf0c8a2ca518b93?s=128

Matt Dancho

February 19, 2020
Tweet

Transcript

  1. With the Quandl API Matt Dancho & David Curry Business

    Science Learning Lab Shiny Crude Oil App (ARIMA)
  2. Learning Lab Structure • Presentation (20 min) • Demo’s (30

    min) • Pro-Tips (15 mins)
  3. Shiny API Series • Lab 28 - Shiny Real Estate

    App ◦ Zillow API • Lab 29 - Shiny Oil & Gas App ◦ Quandl API • Lab 30 - Shiny Finance App ◦ Tidyquant API • Lab 31 - Shiny Marketing App ◦ Google Analytics API
  4. Learning Labs PRO Every 2-Weeks 1-Hour Course Recordings + Code

    + Slack $19/month university.business-science.io Lab 29 - Shiny API Series, Pt 2 Shiny + Quandl + ARIMA for Energy Forecasting Lab 28 - Shiny API Series, Pt 1 Shiny + Zillow for Real Estate Lab 27 - Marketing Series, Pt 4 Google Trends Automation with Shiny Lab 26 - Marketing Series, Pt 3 Customer Journey with Machine Learning Lab 25 - Marketing Series, Pt 2 Attribution with ChannelAttribution Lab 24 - Marketing Series, Pt 1 A/B Testing with Infer Lab 23 - SQL Series SQL with BigQuery & Conversion Funnel Lab 22 - SQL Series SQL for Time Series Continuous Learning Advanced Topics
  5. Agenda • Demo ◦ Shiny App that Forecasts Crude Oil

    (WTI) Prices • Why a Crude Oil ARIMA App? ◦ Helping businesses predict future impacts • Data Science Workflow Concepts ◦ What Impacts Prices? ◦ TS Feat. Engineering ◦ Multivariate ARIMA ◦ Deploying Models into Production • 30-Min Demo ◦ Quandl API ◦ Recipes Prep. ◦ ARIMA Workflow ◦ Shiny 5-Model ARIMA App [LL PRO] • Pro-Tips & Learning Guide
  6. Demo

  7. Why make a WTI Forecast App? Energy

  8. Real Story I used to work for a valve &

    fitting manufacturer Main consumers were Oil & Gas. Product Sales were tied to oil prices. More correctly, sales were tied to Lags in Oil Prices. Since we knew which way oil prices were going, we could predict revenue & product demand.
  9. API + ARIMA + Shiny How an app can help

    Quandl has a vast amount of energy & financial data. • Historical Stock Prices • Energy Data (EIA) • Economic Data (FRED) We can use this data to forecast the WTI Crude Oil Price. Forecasting (NEW)
  10. Benefits Oil & Gas Companies can better forecast revenue

  11. Benefits Product Suppliers can anticipate demand

  12. Benefits Transportation can better estimate fuel costs

  13. How to Forecast WTI Challenging Problem

  14. EIA & Academia https://arxiv.org/pdf/1811.08963.pdf https://www.eia.gov/

  15. EIA & Academia https://arxiv.org/pdf/1811.08963.pdf https://www.eia.gov/finance/markets/crudeoil/ EIA G. Natarajan and A.

    Ashok
  16. Time Series Analysis In 3 minutes

  17. Concept 1: Lag Lags Offsets (shifts) the value 1 or

    more time units.
  18. Concept 2: Lags Plots & Correlation Lag Plots Compare Autocorrelation

    WTI vs Lag(WTI) Cross Correlation WTI vs Lag(Predictor) WTI vs Lag(WTI, n = 1:12)
  19. Concept 3: Predicting with Lags

  20. Crude Oil Forecasting Concepts & Workflow

  21. Workflow Step-By-Step Start Finish 1 2 3 Quandl API Connect

    to Quandl Preprocess & Visualize Data ARIMA Create models that forecast using lagged features Shiny App that productionizes the models
  22. Forecast App

  23. 30-Min Demo Let’s do this!

  24. PRO-TIPS Yeahhhhhh!

  25. Pro-Tip #1: Give Businesses Apps Businesses Need Apps Businesses can’t

    scale reports Businesses can scale apps Apps help decision making
  26. Pro-Tip #2: Leverage Shiny by integrating your analysis Shiny is

    production for your analysis Shiny packages up: Machine Learning Visualizations Interactivity Use Shiny to help decision making
  27. Pro-Tip #3: Say no to reports No https://arxiv.org/pdf/1811.08963.pdf

  28. Pro-Tip #3: ...and say YES to apps. Yes apps.business-science.io

  29. Advanced Visualization Advanced Data Wrangling Advanced Functional Programming & Modeling

    Advanced Machine Learning Visualization Data Cleaning & Manipulation Data Science Algorithms & Iteration Business Reporting Business Analysis with R (DS4B 101-R) Data Science For Business with R (DS4B 201-R) Web Apps & Shiny Developer (DS4B 102-R + DS4B 202A-R) Web Apps Data Science Foundations 7 Weeks Machine Learning & Business Consulting 10 Weeks Web Application Development 12 Weeks -TRACK Project-Based Courses with Business Application Business Science University R-Track 4-Course R-Track System
  30. Key Benefits - Fundamentals - Weeks 1-5 (25 hours of

    Video Lessons) - Data Manipulation (dplyr) - Time series (lubridate) - Text (stringr) - Categorical (forcats) - Visualization (ggplot2) - Programming & Iteration (purrr) - 3 Challenges - Machine Learning - Week 6 (8 hours of Video Lessons) - Clustering (3 hours) - Regression (5 hours) - 2 Challenges - Learn Business Reporting - Week 7 - RMarkdown & plotly - 2 Project Reports: 1. Product Pricing Algo 2. Customer Segmentation Visualization Data Cleaning & Manipulation Business Reporting Business Analysis with R (DS4B 101-R) Data Science Foundations 7 Weeks Data Science Algorithms & Iteration
  31. Key Benefits Understanding the Problem & Preparing Data - Weeks

    1-4 - Project Setup & Framework - Business Understanding / Sizing Problem - Tidy Evaluation - rlang - EDA - Exploring Data -GGally, skimr - Data Preparation - recipes - Correlation Analysis - 3 Challenges Machine Learning - Weeks 5, 6, 7 - H2O AutoML - Modeling Churn - ML Performance - LIME Feature Explanation Return-On-Investment - Weeks 7, 8, 9 - Expected Value Framework - Threshold Optimization - Sensitivity Analysis - Recommendation Algorithm Data Science For Business (DS4B 201-R) Machine Learning & Business Consulting 10 Weeks Advanced Visualization Advanced Data Wrangling Advanced Functional Programming & Modeling Advanced Data Science End-to-End Churn Project
  32. Key Benefits Learn Shiny & Flexdashboard - Build Applications -

    Learn Reactive Programming - Integrate Machine Learning App #1: Predictive Pricing App - Model Product Portfolio - XGBoost Pricing Prediction - Generate new products instantly App #2: Sales Dashboard with Demand Forecasting - Model Demand History - Segment Forecasts by Product & Customer - XGBoost Time Series Forecast - Generate new forecasts instantly Shiny Apps for Business (DS4B 102-R) Web Application Development 4 Weeks Web Apps Machine Learning
  33. Key Benefits Frontend + Backend + Production Deployment Frontend for

    Shiny - Bootstrap Backend for Shiny - MongoDB - Dynamic UI - User Authentication - Store & Write User Data Production Deployment - AWS - EC2 Server - VPC Connection - URL Routing Shiny Apps for Business (DS4B 202A-R) Web Application Development 6 Weeks
  34. -TRACK BUNDLE 15% OFF PROMO Code: learninglabs $127/mo Limited Time

  35. Begin Learning Today university.business-science.io