Anyscale Demo: Machine Learning Application from Dev to Prod (Edward Oakes, Anyscale)

Edward Oakes, Anyscale Software Engineer, presents the "Anyscale Demo: Machine Learning Application from Dev to Prod".
This demo shows the full machine learning-based application lifecycle on Anyscale. You’ll see how Anyscale makes it seamless both to go from developing on your laptop to running on a cluster and to transition from development to serving in production.

Transcript

1. Building End-to-end
   AI Applications
   With Anyscale
   by Edward Oakes
   

   View Slide

2. What do we want to build today?
   A movie recommendation system
   

   View Slide

3. Web
   Server
   User Request
   Recommendations
   What does it take to build this?
   

   View Slide

4. Web
   Server
   Model A
   User Request
   Model B
   Recommendations
   What does it take to build this?
   

   View Slide

5. Web
   Server
   Model A
   User Request
   Model B
   Per-user
   State
   Recommendations
   Custom
   Service
   What does it take to build this?
   

   View Slide

6. Web
   Server
   Model A
   User Request
   Model B
   Per-user
   State
   Recommendations
   Periodic
   Retrainer
   Deploy New
   Models
   User
   Data
   Custom
   Service
   Custom
   Service
   What does it take to build this?
   

   View Slide

7. Recs
   User Request
   Per-user
   State
   Custom
   Service
   Web
   Server
   User Data
   Model A
   Model B
   Production Serving
   What does it take to build this?
   Deploy New
   Models
   Periodic
   Retrainer
   Custom
   Service
   

   View Slide

8. Deploy New
   Models
   Parallel Python
   Processing
   DB
   What does it take to build this?
   Recs
   User Request
   Per-user
   State
   Custom
   Service
   Web
   Server
   User Data
   Model A
   Model B
   Production Serving
   Recs
   Periodic
   Retrainer
   Custom
   Service
   

   View Slide

9. Periodic
   Retrainer
   Custom
   Service
   User Data
   Deploy New
   Models
   Parallel Python
   Processing
   ML Training/HPO
   DB
   What does it take to build this?
   Recs
   User Request
   Per-user
   State
   Custom
   Service
   Web
   Server
   Model A
   Model B
   Production Serving
   

   View Slide

10. Periodic
    Retrainer
    Deploy New
    Models
    Parallel Python
    Processing
    ML Training/HPO
    What does it take to build this?
    DB
    Recs
    User Request
    Per-user
    State
    Custom
    Service
    Web
    Server
    User Data
    Model A
    Model B
    Custom
    Service
    Production Serving
    

    View Slide

11. Distributed computing made simple
    with Ray & Anyscale
    Ray
    Universal Distributed Computing API
    Anyscale
    Laptop experience with the power
    of the cloud
    

    View Slide

12. Recs
    Parallel Python
    Processing
    ML Training/HPO
    How can we build it with Ray + Anyscale?
    DB
    User Request
    Per-user
    State
    Custom
    Service
    Periodic
    Retrainer
    Web
    Server
    User
    Data
    Model A
    Model B
    Custom
    Service
    Production Serving
    Deploy New
    Models
    Ray: Universal Distributed Computing API
    

    View Slide

13. Recs
    Parallel Python
    Processing
    ML Training/HPO
    DB
    How can we build it with Ray + Anyscale?
    User Request
    Per-user
    State
    Custom
    Service
    Periodic
    Retrainer
    Web
    Server
    User
    Data
    Model A
    Model B
    Custom
    Service
    Production Serving
    Deploy New
    Models
    Ray: Universal Distributed Computing API
    

    View Slide

14. Parallel Python
    Processing
    ML Training/HPO
    DB
    How can we build it with Ray + Anyscale?
    Recs
    User Request
    Per-user
    State
    Custom
    Service
    Periodic
    Retrainer
    Web
    Server
    User
    Data
    Model A
    Model B
    Custom
    Service
    Production Serving
    Deploy New
    Models
    Ray: Universal Distributed Computing API
    

    View Slide

15. Recs
    User Request
    Per-user
    State
    Periodic
    Retrainer
    User Data
    Model A
    Model B
    Parallel Python
    Processing
    ML Training/HPO
    DB
    Production Serving
    How can we build it with Ray + Anyscale?
    Deploy New
    Models
    Custom
    Service
    Custom
    Service
    Ray: Universal Distributed Computing API
    

    View Slide

16. How can we build it with Ray + Anyscale?
    Recs
    User Request
    Per-user
    State
    Periodic
    Retrainer
    User Data
    Model A
    Model B
    Parallel Python
    Processing
    ML Training/HPO
    DB
    Production Serving
    Deploy New
    Models
    Ray
    Actor
    Ray
    Actor
    Ray: Universal Distributed Computing API
    

    View Slide

17. Parallel Python
    Processing
    ML Training/HPO
    DB
    Anyscale: Multi-Cloud Native, Universal Computing Platform
    Recs
    User Request
    Per-user
    State
    Ray
    Actor
    Periodic
    Retrainer
    User Data
    Model A
    Model B
    Ray
    Actor
    Production Serving
    Deploy New
    Models
    Ray: Universal Distributed Computing API
    

    View Slide

18. Let’s Build It!
    

    View Slide

19. Let’s Build It - Part 1
    Parallel Python Processing
    

    View Slide

20. Per-user
    State
    Model A
    Recs
    User Request
    Ray
    Actor
    Redeploy Model
    Periodic
    Retrainer
    User Data
    Model B
    Ray
    Actor
    Parallel Python
    Processing
    ML Training/HPO
    DB
    Production Serving
    Anyscale: Multi-Cloud Native, Universal Computing Platform
    Ray: Universal Distributed Computing API
    

    View Slide

21. First Model: Cover Image Similarity
    A movie you liked
    Movies with similar colors
    

    View Slide

22. First Model: Cover Image Similarity
    KNN Search
    A movie you liked
    Movies with similar colors
    

    View Slide

23. First Model: Cover Image Similarity
    KNN Search
    A movie you liked
    Movies with similar colors
    FAISS
    

    View Slide

24. First Model: Cover Image Similarity
    

    View Slide

25. First Model: Cover Image Similarity
    

    View Slide

26. @ray.remote
    First Model: Cover Image Similarity
    

    View Slide

27. No code changes
    @ray.remote
    First Model: Cover Image Similarity
    

    View Slide

28. Let’s Build It - Part 2
    ML Training /HPO
    

    View Slide

29. Model A
    Recs
    User Request
    Per-user
    State
    Ray
    Actor
    Redeploy Model
    Periodic
    Retrainer
    User Data
    Model B
    Ray
    Actor
    Parallel Python
    Processing
    ML Training/HPO
    DB
    Production Serving
    Anyscale: Multi-Cloud Native, Universal Computing Platform
    Ray: Universal Distributed Computing API
    

    View Slide

30. Second Model: Plot Similarity using NLP
    KNN Search
    A movie you liked
    Movies with
    similar plots
    

    View Slide

31. Second Model: Plot Similarity using NLP
    KNN Search
    A movie you liked
    Movies with
    similar plots
    BERT
    

    View Slide

32. Second Model: Plot Similarity using NLP
    KNN Search
    A movie you liked
    Movies with
    similar plots
    BERT
    Ranking Model
    1.
    2.
    3.
    

    View Slide

33. dev-session
    anyscale up
    Recap: Anyscale Serverless Experience
    

    View Slide

34. Recap: Anyscale Serverless Experience
    dev-session
    anyscale exec
    

    View Slide

35. dev-session
    Recap: Anyscale Serverless Experience
    

    View Slide

36. anyscale exec
    dev-session
    Recap: Anyscale Serverless Experience
    

    View Slide

37. Let’s Build It - Part 3
    Production Serving
    

    View Slide

38. Parallel Python
    Processing
    ML Training/HPO
    DB
    Recs
    User Request
    Per-user
    State
    Ray
    Actor
    Periodic
    Retrainer
    User Data
    Model A
    Model B
    Ray
    Actor
    Production Serving
    Deploy New
    Models
    Anyscale: Multi-Cloud Native, Universal Computing Platform
    Ray: Universal Distributed Computing API
    

    View Slide

39. Dynamic
    Model
    Selection
    DB
    Get
    Recommendations
    BERT
    Plot Model
    Periodic
    Retrainer
    Per-user
    State
    Deploy New
    Models
    User
    Click
    Data
    Fine-tuned
    Embeddings
    Color
    Palettes
    Movie
    Cover
    Model
    

    View Slide

40. Dynamic
    Model
    Selection
    DB
    Get
    Recommendations
    BERT
    Plot Model
    Periodic
    Retrainer
    Per-user
    State
    User
    Click
    Data
    Fine-tuned
    Embeddings
    Color
    Palettes
    Movie
    Cover
    Model
    Deploy New
    Models
    

    View Slide

41. Dynamic
    Model
    Selection
    DB
    Get
    Recommendations
    BERT
    Plot Model
    Periodic
    Retrainer
    Per-user
    State
    User
    Click
    Data
    Fine-tuned
    Embeddings
    Color
    Palettes
    Movie
    Cover
    Model
    Deploy New
    Models
    

    View Slide

42. Dynamic
    Model
    Selection
    DB
    HTTP API for
    Recommendations
    BERT
    Plot Model
    Per-user
    State
    Movie Info,
    Fine-tuned
    Embeddings
    Movie
    Info
    Random
    Model
    

    View Slide

43. Woohoo!
    Let’s review what we built together.
    

    View Slide

44. Parallel Python
    Processing
    ML Training/HPO
    DB
    Recs
    User Request
    Per-user
    State
    Ray
    Actor
    Periodic
    Retrainer
    User Data
    Model A
    Model B
    Ray
    Actor
    Production Serving
    Deploy New
    Models
    Anyscale: Multi-Cloud Native, Universal Computing Platform
    Ray: Universal Distributed Computing API
    

    View Slide

45. Anyscale Universal Computing Platform
    Laptop development
    

    View Slide

46. Anyscale Universal Computing Platform
    Serverless experience
    Laptop development
    

    View Slide

47. Anyscale Universal Computing Platform
    Serverless experience
    Laptop development Multi-cloud native
    

    View Slide

48. Thanks for listening!
    48
    ● Visit anyscale.com to sign up for the private beta!
    ● Get in touch on the #4-anyscale channel on the Ray
    Summit slack.
    ● Swing by our Office Hours at 10:55AM on Wednesday or
    10:40AM Thursday.
    

    View Slide