The 2021 Year in Review of Ray

We will reflect back on Ray’s major milestones, Ray’s ecosystems of ML native and integrated libraries, and community growth and contributions.

Transcript

1. Welcome to the Bay Area Ray Meetup! Jan 19, 2022
2. Agenda: Virtual Meetup 6:00 PM - Kickoff Welcome remarks & agenda by Jules Damji 6:05 PM - “Year in Review: State of Ray in 2021” by Robert Nishihara 6:15 PM - “What’s New in Ray 1.9 and Beyond” by Zhe Zhang 6:35 PM - “Unifying Preprocessing and Training at Scale with Ray Datasets" by Alex Wu & Clark Zinzow 7:15 PM - "DeltaCAT: A Scalable Data Catalog for Ray Datasets" by Patrick Ames, Amazon
3. Year in Review: State of Ray in 2021 Robert Nishihara CEO and Co-Founder, Anyscale
4. An Amazing Year for Ray Vibrant Community Richer Ecosystem Inspiring User Stories
5. 2017 2018 2019 2020 2021 590+ contributors 597 200 400 600 Community
6. Community Releases, Commits, Slack Members 14 10K+ Community 3.5K
7. Vibrant Community Richer Ecosystem Inspiring User Stories An Amazing Year for Ray
8. Native Libraries 3rd Party Libraries Your app here! Ecosystem - the big picture! Universal framework for distributed computing Run anywhere Library + app ecosystem
9. New and improved integrations Ecosystem and more ...
10. 5X speed up with Ray XGBoost and more ...
11. “Ray will play an increasingly important role in bringing much needed common infrastructure and standardization to the production machine learning ecosystem, both within Uber and the industry at large.” Horovod and more ...
12. 13X speed up with Ray Dask and more ...
13. Ray is the tool of choice for scaling libraries Ecosystem and more ...
14. Vibrant Community Richer Ecosystem Inspiring User Stories An Amazing Year for Ray
15. Diverse users … with diverse use cases
16. Making Boats Fly with AI Mckinsey | QuantumBlack Australia
17. Scaling Ecosystem Restoration Dendra Systems
18. Large Scale ML Platforms Uber, Shopify, and more
19. Vibrant Community Richer Ecosystem Inspiring User Stories An Amazing Year for Ray
20. What’s New in Ray 1.9 & Beyond Zhe Zhang Head of Open Source Engineering @Anyscale
21. Overall Themes Core: Reliable and stable at large scale Libraries: Easy-to-use high level libraries for production workloads Deployment: Simple and clear paths for deploying clusters and code
22. Ray Core – Stable at Large Scale Pluggable GCS backend
23. Ray Core – Stable at Large Scale Pluggable GCS backend Redis GCS Raylet Worker Fetch GCS location via Redis Redis Pubsub KV via GCS Redis backs GCS KV and internal data tables
24. Ray Core – Stable at Large Scale - Object lifetime mgmt - PB level data processing - Rock solid scheduler https://github.com/orgs/ray-project/projects/10/views/5
25. Libraries: Easy / Production High level plan Dataset Workflow Train Serve Pipelines 1.10 Beta Alpha Beta Experimental 1.11 / 1.12 GA Beta (❤ your input!) Alpha/Beta
26. Ray.data
27. Ray Workflows
28. Deployment – Clear and Simple Ray Client: Interactive development Job Submission: Production jobs runtime_env: Dependencies and package
29. Deployment / Language Ecosystem Beta in 1.10! Active integration with KubeRay Continuously improving
30. Start learning Ray and contributing … Getting Started: Documentation (docs.ray.io) Quick start example, reference guides, etc Join Ray Meetup Revive in Jan 2022. Publish recording to the members https://www.meetup.com/Bay-Area-Ray-Meetup/ Forums (discuss.ray.io) Learn / share with broader Ray community, including core team Ray Slack Connect with the Ray team and community Social Media (@raydistrtibuted, @anyscalecompute) Follow us on Twitter and linkedIn GitHub Check out sources, file an issue, become a contributor, give us a Star :) https://github.com/ray-project/ray
31. Thank you!