Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The 2021 Year in Review of Ray

The 2021 Year in Review of Ray

We will reflect back on Ray’s major milestones, Ray’s ecosystems of ML native and integrated libraries, and community growth and contributions.



January 21, 2022

More Decks by Anyscale

Other Decks in Technology


  1. Welcome to the Bay Area Ray Meetup! Jan 19, 2022

  2. Agenda: Virtual Meetup 6:00 PM - Kickoff Welcome remarks &

    agenda by Jules Damji 6:05 PM - “Year in Review: State of Ray in 2021” by Robert Nishihara 6:15 PM - “What’s New in Ray 1.9 and Beyond” by Zhe Zhang 6:35 PM - “Unifying Preprocessing and Training at Scale with Ray Datasets" by Alex Wu & Clark Zinzow 7:15 PM - "DeltaCAT: A Scalable Data Catalog for Ray Datasets" by Patrick Ames, Amazon
  3. Year in Review: State of Ray in 2021 Robert Nishihara

    CEO and Co-Founder, Anyscale
  4. An Amazing Year for Ray Vibrant Community Richer Ecosystem Inspiring

    User Stories
  5. 2017 2018 2019 2020 2021 590+ contributors 597 200 400

    600 Community
  6. Community Releases, Commits, Slack Members 14 10K+ Community 3.5K

  7. Vibrant Community Richer Ecosystem Inspiring User Stories An Amazing Year

    for Ray
  8. Native Libraries 3rd Party Libraries Your app here! Ecosystem -

    the big picture! Universal framework for distributed computing Run anywhere Library + app ecosystem
  9. New and improved integrations Ecosystem and more ...

  10. 5X speed up with Ray XGBoost and more ...

  11. “Ray will play an increasingly important role in bringing much

    needed common infrastructure and standardization to the production machine learning ecosystem, both within Uber and the industry at large.” Horovod and more ...
  12. 13X speed up with Ray Dask and more ...

  13. Ray is the tool of choice for scaling libraries Ecosystem

    and more ...
  14. Vibrant Community Richer Ecosystem Inspiring User Stories An Amazing Year

    for Ray
  15. Diverse users … with diverse use cases

  16. Making Boats Fly with AI Mckinsey | QuantumBlack Australia

  17. Scaling Ecosystem Restoration Dendra Systems

  18. Large Scale ML Platforms Uber, Shopify, and more

  19. Vibrant Community Richer Ecosystem Inspiring User Stories An Amazing Year

    for Ray
  20. What’s New in Ray 1.9 & Beyond Zhe Zhang Head

    of Open Source Engineering @Anyscale
  21. Overall Themes Core: Reliable and stable at large scale Libraries:

    Easy-to-use high level libraries for production workloads Deployment: Simple and clear paths for deploying clusters and code
  22. Ray Core – Stable at Large Scale Pluggable GCS backend

  23. Ray Core – Stable at Large Scale Pluggable GCS backend

    Redis GCS Raylet Worker Fetch GCS location via Redis Redis Pubsub KV via GCS Redis backs GCS KV and internal data tables
  24. Ray Core – Stable at Large Scale - Object lifetime

    mgmt - PB level data processing - Rock solid scheduler https://github.com/orgs/ray-project/projects/10/views/5
  25. Libraries: Easy / Production High level plan Dataset Workflow Train

    Serve Pipelines 1.10 Beta Alpha Beta Experimental 1.11 / 1.12 GA Beta (❤ your input!) Alpha/Beta
  26. Ray.data

  27. Ray Workflows

  28. Deployment – Clear and Simple Ray Client: Interactive development Job

    Submission: Production jobs runtime_env: Dependencies and package
  29. Deployment / Language Ecosystem Beta in 1.10! Active integration with

    KubeRay Continuously improving
  30. Start learning Ray and contributing … Getting Started: Documentation (docs.ray.io)

    Quick start example, reference guides, etc Join Ray Meetup Revive in Jan 2022. Publish recording to the members https://www.meetup.com/Bay-Area-Ray-Meetup/ Forums (discuss.ray.io) Learn / share with broader Ray community, including core team Ray Slack Connect with the Ray team and community Social Media (@raydistrtibuted, @anyscalecompute) Follow us on Twitter and linkedIn GitHub Check out sources, file an issue, become a contributor, give us a Star :) https://github.com/ray-project/ray
  31. Thank you!