Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Faster time series forecasting using Ray and Anyscale

Faster time series forecasting using Ray and Anyscale

Forecasting is an important part of running any business. The “embarrassingly parallel” pattern of distributed computing works with time series algorithms such as ARIMA, Prophet, and Neural Prophet. For global deep learning algorithms, both data and model parallelism become simple and easy with Ray.

In this webinar, we will demo both patterns. For the deep learning algorithm, we will show how to take Google’s Temporal Fusion Transformer, available in PyTorch Forecasting, and turn training and inference into distributed applications using Ray. Ray code runs in parallel across local laptop cores. The exact same code can run in parallel across any cluster in any cloud too.

Af07bbf978a0989644b039ae6b8904a5?s=128

Anyscale
PRO

January 20, 2022
Tweet

More Decks by Anyscale

Other Decks in Technology

Transcript

  1. Faster forecasting using Ray and Anyscale Christy Bergman, @cbergman Developer

    Advocate, Anyscale Amog Kamsetty Software Engineer Ray Train, Anyscale
  2. • Forecasting: Introduction and challenges • Why Ray? • Ray

    overview and integration with PyTorch Lightning • Forecasting with Ray ◦ Demo on laptop (Ray) and cloud (Anyscale) • Q & A Overview
  3. Typically a forecast is a time series that predicts both

    the quantity and timing of something. Example: a demand forecast predicts the quantity and timing of sales quantities for a particular product. What is a Forecast
  4. Statistical algorithms: Individually train a different model for each time

    series. - ARIMA, Prophet, Neural Prophet Forecasting existing tools Deep learning algorithms: Train a global model for all time series at once. - RNN, CNN, Temporal Fusion Transformer
  5. Statistical algorithms: Individually train a different model for each time

    series. - ARIMA, Prophet, Neural Prophet Forecasting existing tools Deep learning algorithms: Train a global model for all time series at once. - RNN, CNN, TemporalFusionTransformer
  6. Statistical algorithms: Individually train a different model for each time

    series. Accelerate with distributed computing Deep learning algorithms: Train a global model for all time series at once Same data / same functions with fully sharded fully parallel data and gradients (e.g. training a deep neural network global model) Different data / different functions (e.g. individually-trained time-series)
  7. Statistical algorithms: Individually train a different model for each time

    series. Accelerate with distributed computing Deep learning algorithms: Train a global model for all time series at once Same data / same functions with fully sharded fully parallel data and gradients (e.g. training a deep neural network global model) Different data / different functions (e.g. individually-trained time-series)
  8. - Machine learning is pervasive in every domain - Distributed

    machine learning is becoming a necessity - Distributed systems is notoriously hard Ray’s vision: Make distributed computing accessible to every developer Why Ray?
  9. 35x every 18 m onths 2020 GPT-3 Moore’s Law (2x

    every 18 months) CPU https://openai.com/blog/ai-and-compute/ GPU* TPU * Compute demand growing faster than supply
  10. 35x every 18 m onths 2020 GPT-3 Specialized hardware is

    also not enough Moore’s Law (2x every 18 months) CPU https://openai.com/blog/ai-and-compute/ GPU* TPU * No way out but to distribute!
  11. Overview Ray & integration with PyTorch Lightning

  12. The Ray Layered Cake and Ecosystem Datasets Workflows Run anywhere

    Universal framework for distributed computing Library + app ecosystem
  13. Native Libraries 3rd Party Libraries Your app here! Universal framework

    for distributed computing Run anywhere Library + app ecosystem
  14. Native Libraries 3rd Party Libraries Your app here! Universal framework

    for distributed computing Run anywhere Library + app ecosystem
  15. Native Libraries Your app here! Universal framework for distributed computing

    Run anywhere Library + app ecosystem Ray Lightning https://github.com/ray-project/ray_lightning
  16. Universal framework for distributed computing Run anywhere Library + app

    ecosystem Ray Lightning https://github.com/ray-project/ray_lightning
  17. Forecasting with Ray

  18. Example Parallelism Patterns Forecasting Algorithms on Ray Train statistical models

    Train global DL models Same data / same functions with fully sharded fully parallel data and gradients (e.g. training a deep neural network global model)
  19. Accelerating statistical algorithms with Ray core An example with Prophet

    • Convert already-existing Python functions & classes to remote Ray Tasks and Actors in 6 steps: 1. install and import ray and anyscale 2. ray.init(), 3. decorate with @ray.remote 4. ray.put(data) 5. call distributed remote tasks with .remote() 6. ray.get() results • Same concepts can be used for ARIMA ….
  20. Accelerating statistical algorithms with Ray core An example with Prophet

    Step 1. Install and import ray and anyscale
  21. Prophet Train Function Prophet Inference fn def train_prophet(df): # split

    df into train/test # fit prophet on train data return train_df,test_df,model # read data into pandas df_raw = read(file1.parquet) # train on train data df_train, df_test, model =\ train_prophet(df_raw) def inf_prophet(model, test_df): # get n_periods from test_df # create forecast forecast = model.predict(...) return forecast # inference on test data forecast = \ inf_prophet(model, df_test)
  22. Prophet Train Function Prophet Inference fn @ray.remote(num_returns=3) def train_prophet(df): #

    split df into train/test # fit prophet on train data return train_df,test_df,model # read data into pandas df_raw = read(file1.parquet) # train on train data df_train, df_test, model =\ train_prophet(df_raw) @ray.remote def inf_prophet(model, test_df): # get n_periods from test_df # create forecast forecast = model.predict(...) return forecast # inference on test data forecast = \ inf_prophet(model, df_test) 3 3 Step 3: decorate with @ray.remote
  23. Prophet Train Function Prophet Inference fn @ray.remote(num_returns=3) def train_prophet(df): #

    split df into train/test # fit prophet on train data return train_df,test_df,model # read data into pandas df_raw = read(file1.parquet) # distribute data input_data_ref = ray.put(df_raw) # train on train data df_train, df_test, model =\ train_prophet.remote(df_raw) @ray.remote def inf_prophet(model, test_df): # get n_periods from test_df # create forecast forecast = model.predict(...) return forecast # inference on test data forecast = inf_prophet.remote( model, df_test) 4 Step 4: ray.put(data) onto each compute node
  24. Prophet Train Function Prophet Inference fn @ray.remote(num_returns=3) def train_prophet(df): #

    split df into train/test # fit prophet on train data return train_df,test_df,model # read data into pandas df_raw = read(file1.parquet) # distribute data input_data_ref = ray.put(df_raw) # train on train data df_train, df_test, model =\ train_prophet.remote(df_raw) @ray.remote def inf_prophet(model, test_df): # get n_periods from test_df # create forecast forecast = model.predict(...) return forecast # inference on test data forecast = inf_prophet.remote( model, df_test) 5 5 Step 5: call distributed remote tasks
  25. Prophet Train Function Prophet Inference fn @ray.remote(num_returns=3) def train_prophet(df): #

    split df into train/test # fit prophet on train data return train_df,test_df,model # read data into pandas df_raw = read(file1.parquet) # distribute data input_data_ref = ray.put(df_raw) # train on train data df_train, df_test, model =\ train_prophet.remote(df_raw) @ray.remote def inf_prophet(model, test_df): # get n_periods from test_df # create forecast forecast = model.predict(...) return forecast # inference on test data forecast_ray_obj_ref = \ inf_prophet.remote( model, df_test) forecast = \ ray.get(forecast_ray_obj_ref) 6 Step 6: ray.get() results
  26. Prophet Main function The .remote() tasks are scheduled in parallel

    asynchronously file1 file1 Node 1 Node 2 train_prophet Blue variables are Object IDs (similar to futures) train_prophet inf_prophet inf_prophet id1 id2 import ray # initialize ray ray.init() # train on train data df_train, df_test, model =\ train_prophet.remote(df_raw) # inference on test data forecast_ray_obj_ref = \ inf_prophet.remote( model, df_test) forecast = \ ray.get(forecast_ray_obj_ref) 4 1 4 Step 4: call distributed remote tasks 2
  27. Prophet Main function file1 file1 Node 1 Node 2 train_prophet

    Blue variables are Object IDs (similar to futures) train_prophet inf_prophet inf_prophet id1 id2 import ray # initialize ray ray.init() # train on train data df_train, df_test, model =\ train_prophet.remote(df_raw) # inference on test data forecast_ray_obj_ref = \ inf_prophet.remote( model, df_test) forecast = \ ray.get(forecast_ray_obj_ref) 5 ray.get() blocks until all remote task results are available Step 5: ray.get() results
  28. ARIMA Train function ARIMA Inference fn @ray.remote(num_returns=3) def train_arima(df): #

    split df into train/test # fit prophet on train data # pickle the ARIMA model return train_df,test_df,model # read data into pandas df_raw = read(file1.parquet) # train on train data df_train, df_test, model =\ train_arima.remote(df_raw) @ray.remote def inf_arima(model, test_df): # get n_periods from test_df # unpickle ARIMA model # create forecast forecast = model.predict(...) return forecast # inference on test data forecast_ray_obj_ref = \ inf_arima.remote( model, df_test) forecast = \ ray.get(forecast_ray_obj_ref) 3’ 3’ 3 3 4 4 5
  29. Demo: Prophet + Ray https://tinyurl.com/anyscale-forecasting

  30. Example Parallelism Patterns Forecasting Algorithms on Ray Train statistical models

    Train global DL models Same data / same functions with fully sharded fully parallel data and gradients (e.g. training a deep neural network global model)
  31. Accelerating deep learning forecasting with Ray Example with PyTorch Lightning

    / Ray Lightning • Convert already-existing Python functions & classes to remote Ray Tasks and Actors in 4 easy steps: 1. pip install and import Ray, Ray Lightning, and Anyscale 2. ray.init() 3. Create a RayPlugin object 4. Modify the PyTorch Lightning Trainer to use Ray library plugin • For regular distributed PyTorch or TensorFlow2 use Ray Train instead. Those integrations use the same concepts shown here, but they are built-in to Ray Train’s Trainer.
  32. import ray from ray_lightning import RayPlugin # initialize ray and

    plugin ray.init() plugin = RayPlugin() # convert data to PyTorch tensors # modify PyTorch Lightning trainer model, trainer = \ train_model(ptf_train_data, plugin) # fit PyTorch Forecasting model trainer.fit(model, pt_train_loader, pt_val_loader) # get best model from the trainer # inference on validation data predictions = best_model.predict(pt_val_loader) PyTorch Main function Node 1 Node 2 Same data parallel sharded, same functions, shared gradients 2 4 shard1 train_model gradients shard2 train_model gradients gradients 3 1
  33. Demo: PyTorch + Ray https://tinyurl.com/anyscale-forecasting

  34. Takeaways • Distributed computing is a necessity and norm •

    Ray’s vision: make distributed programming simple using core @ray.remote or Ray Libraries • Ray Core: Easily convert already-existing Python functions & classes to remote Ray Tasks and Actors in 6 steps: install, .init(), decorate, ray.put(data), .remote(), .get() • Ray Libraries: Convert existing ML or DL frameworks using Ray Libraries, example Ray Lightning in 4 steps: install, .init(), plugin, modify trainer to use plugin.
  35. What’s Next?

  36. Upcoming Anyscale events and resources Events (anyscale.com) - Meetup tomorrow

    online, Jan 19th, 6pm PST - Weekly Intro demo / office hours. Thursdays alternating 9am and 2pm PST - Approx monthly webinars on different AI/ML workloads, like this webinar Resources - Ray doc pages (docs.ray.io) - Forums to learn/share with broader Ray community (discuss.ray.io) - Ray Slack (ray.io/community) - Follow us on Twitter and linkedin (@raydistrtibuted, @anyscalecompute) - Github: Check out sources, file an issue, become a contributor, give us a Star :) (github.com/ray-project/ray)
  37. Thank you @raydistributed @anyscalecompute