Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Anyscale Demo: Machine Learning Application from Dev to Prod (Edward Oakes, Anyscale)

Anyscale Demo: Machine Learning Application from Dev to Prod (Edward Oakes, Anyscale)

Edward Oakes, Anyscale Software Engineer, presents the "Anyscale Demo: Machine Learning Application from Dev to Prod".
This demo shows the full machine learning-based application lifecycle on Anyscale. You’ll see how Anyscale makes it seamless both to go from developing on your laptop to running on a cluster and to transition from development to serving in production.

Af07bbf978a0989644b039ae6b8904a5?s=128

Anyscale
PRO

July 14, 2021
Tweet

Transcript

  1. Building End-to-end AI Applications With Anyscale by Edward Oakes

  2. What do we want to build today? A movie recommendation

    system
  3. Web Server User Request Recommendations What does it take to

    build this?
  4. Web Server Model A User Request Model B Recommendations What

    does it take to build this?
  5. Web Server Model A User Request Model B Per-user State

    Recommendations Custom Service What does it take to build this?
  6. Web Server Model A User Request Model B Per-user State

    Recommendations Periodic Retrainer Deploy New Models User Data Custom Service Custom Service What does it take to build this?
  7. Recs User Request Per-user State Custom Service Web Server User

    Data Model A Model B Production Serving What does it take to build this? Deploy New Models Periodic Retrainer Custom Service
  8. Deploy New Models Parallel Python Processing DB What does it

    take to build this? Recs User Request Per-user State Custom Service Web Server User Data Model A Model B Production Serving Recs Periodic Retrainer Custom Service
  9. Periodic Retrainer Custom Service User Data Deploy New Models Parallel

    Python Processing ML Training/HPO DB What does it take to build this? Recs User Request Per-user State Custom Service Web Server Model A Model B Production Serving
  10. Periodic Retrainer Deploy New Models Parallel Python Processing ML Training/HPO

    What does it take to build this? DB Recs User Request Per-user State Custom Service Web Server User Data Model A Model B Custom Service Production Serving
  11. Distributed computing made simple with Ray & Anyscale Ray Universal

    Distributed Computing API Anyscale Laptop experience with the power of the cloud
  12. Recs Parallel Python Processing ML Training/HPO How can we build

    it with Ray + Anyscale? DB User Request Per-user State Custom Service Periodic Retrainer Web Server User Data Model A Model B Custom Service Production Serving Deploy New Models Ray: Universal Distributed Computing API
  13. Recs Parallel Python Processing ML Training/HPO DB How can we

    build it with Ray + Anyscale? User Request Per-user State Custom Service Periodic Retrainer Web Server User Data Model A Model B Custom Service Production Serving Deploy New Models Ray: Universal Distributed Computing API
  14. Parallel Python Processing ML Training/HPO DB How can we build

    it with Ray + Anyscale? Recs User Request Per-user State Custom Service Periodic Retrainer Web Server User Data Model A Model B Custom Service Production Serving Deploy New Models Ray: Universal Distributed Computing API
  15. Recs User Request Per-user State Periodic Retrainer User Data Model

    A Model B Parallel Python Processing ML Training/HPO DB Production Serving How can we build it with Ray + Anyscale? Deploy New Models Custom Service Custom Service Ray: Universal Distributed Computing API
  16. How can we build it with Ray + Anyscale? Recs

    User Request Per-user State Periodic Retrainer User Data Model A Model B Parallel Python Processing ML Training/HPO DB Production Serving Deploy New Models Ray Actor Ray Actor Ray: Universal Distributed Computing API
  17. Parallel Python Processing ML Training/HPO DB Anyscale: Multi-Cloud Native, Universal

    Computing Platform Recs User Request Per-user State Ray Actor Periodic Retrainer User Data Model A Model B Ray Actor Production Serving Deploy New Models Ray: Universal Distributed Computing API
  18. Let’s Build It!

  19. Let’s Build It - Part 1 Parallel Python Processing

  20. Per-user State Model A Recs User Request Ray Actor Redeploy

    Model Periodic Retrainer User Data Model B Ray Actor Parallel Python Processing ML Training/HPO DB Production Serving Anyscale: Multi-Cloud Native, Universal Computing Platform Ray: Universal Distributed Computing API
  21. First Model: Cover Image Similarity A movie you liked Movies

    with similar colors
  22. First Model: Cover Image Similarity KNN Search A movie you

    liked Movies with similar colors
  23. First Model: Cover Image Similarity KNN Search A movie you

    liked Movies with similar colors FAISS
  24. First Model: Cover Image Similarity

  25. First Model: Cover Image Similarity

  26. @ray.remote First Model: Cover Image Similarity

  27. No code changes @ray.remote First Model: Cover Image Similarity

  28. Let’s Build It - Part 2 ML Training /HPO

  29. Model A Recs User Request Per-user State Ray Actor Redeploy

    Model Periodic Retrainer User Data Model B Ray Actor Parallel Python Processing ML Training/HPO DB Production Serving Anyscale: Multi-Cloud Native, Universal Computing Platform Ray: Universal Distributed Computing API
  30. Second Model: Plot Similarity using NLP KNN Search A movie

    you liked Movies with similar plots
  31. Second Model: Plot Similarity using NLP KNN Search A movie

    you liked Movies with similar plots BERT
  32. Second Model: Plot Similarity using NLP KNN Search A movie

    you liked Movies with similar plots BERT Ranking Model 1. 2. 3.
  33. dev-session anyscale up Recap: Anyscale Serverless Experience

  34. Recap: Anyscale Serverless Experience dev-session anyscale exec

  35. dev-session Recap: Anyscale Serverless Experience

  36. anyscale exec dev-session Recap: Anyscale Serverless Experience

  37. Let’s Build It - Part 3 Production Serving

  38. Parallel Python Processing ML Training/HPO DB Recs User Request Per-user

    State Ray Actor Periodic Retrainer User Data Model A Model B Ray Actor Production Serving Deploy New Models Anyscale: Multi-Cloud Native, Universal Computing Platform Ray: Universal Distributed Computing API
  39. Dynamic Model Selection DB Get Recommendations BERT Plot Model Periodic

    Retrainer Per-user State Deploy New Models User Click Data Fine-tuned Embeddings Color Palettes Movie Cover Model
  40. Dynamic Model Selection DB Get Recommendations BERT Plot Model Periodic

    Retrainer Per-user State User Click Data Fine-tuned Embeddings Color Palettes Movie Cover Model Deploy New Models
  41. Dynamic Model Selection DB Get Recommendations BERT Plot Model Periodic

    Retrainer Per-user State User Click Data Fine-tuned Embeddings Color Palettes Movie Cover Model Deploy New Models
  42. Dynamic Model Selection DB HTTP API for Recommendations BERT Plot

    Model Per-user State Movie Info, Fine-tuned Embeddings Movie Info Random Model
  43. Woohoo! Let’s review what we built together.

  44. Parallel Python Processing ML Training/HPO DB Recs User Request Per-user

    State Ray Actor Periodic Retrainer User Data Model A Model B Ray Actor Production Serving Deploy New Models Anyscale: Multi-Cloud Native, Universal Computing Platform Ray: Universal Distributed Computing API
  45. Anyscale Universal Computing Platform Laptop development

  46. Anyscale Universal Computing Platform Serverless experience Laptop development

  47. Anyscale Universal Computing Platform Serverless experience Laptop development Multi-cloud native

  48. Thanks for listening! 48 • Visit anyscale.com to sign up

    for the private beta! • Get in touch on the #4-anyscale channel on the Ray Summit slack. • Swing by our Office Hours at 10:55AM on Wednesday or 10:40AM Thursday.