Anyscale Demo: Machine Learning Application from Dev to Prod (Edward Oakes, Anyscale)

Edward Oakes, Anyscale Software Engineer, presents the "Anyscale Demo: Machine Learning Application from Dev to Prod".
This demo shows the full machine learning-based application lifecycle on Anyscale. You’ll see how Anyscale makes it seamless both to go from developing on your laptop to running on a cluster and to transition from development to serving in production.

Transcript

1. Building End-to-end AI Applications With Anyscale by Edward Oakes
2. What do we want to build today? A movie recommendation system
3. Web Server User Request Recommendations What does it take to build this?
4. Web Server Model A User Request Model B Recommendations What does it take to build this?
5. Web Server Model A User Request Model B Per-user State Recommendations Custom Service What does it take to build this?
6. Web Server Model A User Request Model B Per-user State Recommendations Periodic Retrainer Deploy New Models User Data Custom Service Custom Service What does it take to build this?
7. Recs User Request Per-user State Custom Service Web Server User Data Model A Model B Production Serving What does it take to build this? Deploy New Models Periodic Retrainer Custom Service
8. Deploy New Models Parallel Python Processing DB What does it take to build this? Recs User Request Per-user State Custom Service Web Server User Data Model A Model B Production Serving Recs Periodic Retrainer Custom Service
9. Periodic Retrainer Custom Service User Data Deploy New Models Parallel Python Processing ML Training/HPO DB What does it take to build this? Recs User Request Per-user State Custom Service Web Server Model A Model B Production Serving
10. Periodic Retrainer Deploy New Models Parallel Python Processing ML Training/HPO What does it take to build this? DB Recs User Request Per-user State Custom Service Web Server User Data Model A Model B Custom Service Production Serving
11. Distributed computing made simple with Ray & Anyscale Ray Universal Distributed Computing API Anyscale Laptop experience with the power of the cloud
12. Recs Parallel Python Processing ML Training/HPO How can we build it with Ray + Anyscale? DB User Request Per-user State Custom Service Periodic Retrainer Web Server User Data Model A Model B Custom Service Production Serving Deploy New Models Ray: Universal Distributed Computing API
13. Recs Parallel Python Processing ML Training/HPO DB How can we build it with Ray + Anyscale? User Request Per-user State Custom Service Periodic Retrainer Web Server User Data Model A Model B Custom Service Production Serving Deploy New Models Ray: Universal Distributed Computing API
14. Parallel Python Processing ML Training/HPO DB How can we build it with Ray + Anyscale? Recs User Request Per-user State Custom Service Periodic Retrainer Web Server User Data Model A Model B Custom Service Production Serving Deploy New Models Ray: Universal Distributed Computing API
15. Recs User Request Per-user State Periodic Retrainer User Data Model A Model B Parallel Python Processing ML Training/HPO DB Production Serving How can we build it with Ray + Anyscale? Deploy New Models Custom Service Custom Service Ray: Universal Distributed Computing API
16. How can we build it with Ray + Anyscale? Recs User Request Per-user State Periodic Retrainer User Data Model A Model B Parallel Python Processing ML Training/HPO DB Production Serving Deploy New Models Ray Actor Ray Actor Ray: Universal Distributed Computing API
17. Parallel Python Processing ML Training/HPO DB Anyscale: Multi-Cloud Native, Universal Computing Platform Recs User Request Per-user State Ray Actor Periodic Retrainer User Data Model A Model B Ray Actor Production Serving Deploy New Models Ray: Universal Distributed Computing API
18. Let’s Build It!
19. Let’s Build It - Part 1 Parallel Python Processing
20. Per-user State Model A Recs User Request Ray Actor Redeploy Model Periodic Retrainer User Data Model B Ray Actor Parallel Python Processing ML Training/HPO DB Production Serving Anyscale: Multi-Cloud Native, Universal Computing Platform Ray: Universal Distributed Computing API
21. First Model: Cover Image Similarity A movie you liked Movies with similar colors
22. First Model: Cover Image Similarity KNN Search A movie you liked Movies with similar colors
23. First Model: Cover Image Similarity KNN Search A movie you liked Movies with similar colors FAISS
24. First Model: Cover Image Similarity
25. First Model: Cover Image Similarity
26. @ray.remote First Model: Cover Image Similarity
27. No code changes @ray.remote First Model: Cover Image Similarity
28. Let’s Build It - Part 2 ML Training /HPO
29. Model A Recs User Request Per-user State Ray Actor Redeploy Model Periodic Retrainer User Data Model B Ray Actor Parallel Python Processing ML Training/HPO DB Production Serving Anyscale: Multi-Cloud Native, Universal Computing Platform Ray: Universal Distributed Computing API
30. Second Model: Plot Similarity using NLP KNN Search A movie you liked Movies with similar plots
31. Second Model: Plot Similarity using NLP KNN Search A movie you liked Movies with similar plots BERT
32. Second Model: Plot Similarity using NLP KNN Search A movie you liked Movies with similar plots BERT Ranking Model 1. 2. 3.
33. dev-session anyscale up Recap: Anyscale Serverless Experience
34. Recap: Anyscale Serverless Experience dev-session anyscale exec
35. dev-session Recap: Anyscale Serverless Experience
36. anyscale exec dev-session Recap: Anyscale Serverless Experience
37. Let’s Build It - Part 3 Production Serving
38. Parallel Python Processing ML Training/HPO DB Recs User Request Per-user State Ray Actor Periodic Retrainer User Data Model A Model B Ray Actor Production Serving Deploy New Models Anyscale: Multi-Cloud Native, Universal Computing Platform Ray: Universal Distributed Computing API
39. Dynamic Model Selection DB Get Recommendations BERT Plot Model Periodic Retrainer Per-user State Deploy New Models User Click Data Fine-tuned Embeddings Color Palettes Movie Cover Model
40. Dynamic Model Selection DB Get Recommendations BERT Plot Model Periodic Retrainer Per-user State User Click Data Fine-tuned Embeddings Color Palettes Movie Cover Model Deploy New Models
41. Dynamic Model Selection DB Get Recommendations BERT Plot Model Periodic Retrainer Per-user State User Click Data Fine-tuned Embeddings Color Palettes Movie Cover Model Deploy New Models
42. Dynamic Model Selection DB HTTP API for Recommendations BERT Plot Model Per-user State Movie Info, Fine-tuned Embeddings Movie Info Random Model
43. Woohoo! Let’s review what we built together.
44. Parallel Python Processing ML Training/HPO DB Recs User Request Per-user State Ray Actor Periodic Retrainer User Data Model A Model B Ray Actor Production Serving Deploy New Models Anyscale: Multi-Cloud Native, Universal Computing Platform Ray: Universal Distributed Computing API
45. Anyscale Universal Computing Platform Laptop development
46. Anyscale Universal Computing Platform Serverless experience Laptop development
47. Anyscale Universal Computing Platform Serverless experience Laptop development Multi-cloud native
48. Thanks for listening! 48 • Visit anyscale.com to sign up for the private beta! • Get in touch on the #4-anyscale channel on the Ray Summit slack. • Swing by our Office Hours at 10:55AM on Wednesday or 10:40AM Thursday.