Ray and Anyscale: An Optimization Journey (Nachi Mehta, OXW.io)

Having a trained model that can classify is great, but turning that model into a scalable server presents some unique challenges. Ray Serve on Anyscale knocks out each of these challenges for a simple end-to-end experience. In this talk, which is for people new to Ray, we present how quickly you can put a model serving application into production, integrate with other backend services, and scale it up to respond to demand.

Transcript

1. An Optimization Journey Prepared by OXW Ray and Anyscale
2. What was the first thing we tried First Iteration 03 Outline Who we are A quick overview of who we are 01 Problem Statement What are we trying to accomplish 02 Second iteration The Ray object store 04 Third iteration How we leveraged Ray’s Actor model to scale 05
3. Who we are Nachi Mehta Co-Founder at OXW.io Charles Greer Solutions Architect at Anyscale
4. Special Thanks Bill Chambers Head of Product at Anyscale Simon Mo Software Engineer at Anyscale Archit Kulkarni Software Engineer at Anyscale
5. Deliver a scalable machine learning application that predicts product affinity based on user engagement history. Must be modular in design so it’s not tightly coupled with existing code base, allowing for changes to the model in the future without disrupting business logic. Problem Statement
6. Architecture Business Logic Application user engagement data elasticsearch Anyscale Cluster
7. Problem The Ray Object Store allows us to store, in memory, across the cluster, any serializable data we’d like. In this case, our engagement data fetched from elasticsearch. Our scalability is tied to an external data source (elasticsearch). We want to create an abstraction that allows us to capture the required data, and then scale processing independently Ray Object Store
8. Ray Actor Model elasticsearch recommender backend Data Actor Anyscale cluster
9. Infinite Scaling recommender backend Data Actor Anyscale cluster Recommendation API Scaler Backend Scaling hook
10. oxw.io OXW.io works with companies to transform diverse and disparate data into meaningful and actionable business information through the strategic application of data science principles and technologies. nachi@oxw.io
11. anyscale.com Anyscale was founded by the creators of Ray, an open source project from the UC Berkeley RISELab. Anyscale enables developers of all skill levels to easily build applications that run at any scale, from a laptop to a data center. cgreer@anyscale.com