Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The 2021 Year in Review of Ray

Anyscale
January 21, 2022

The 2021 Year in Review of Ray

We will reflect back on Ray’s major milestones, Ray’s ecosystems of ML native and integrated libraries, and community growth and contributions.

Anyscale

January 21, 2022
Tweet

More Decks by Anyscale

Other Decks in Technology

Transcript

  1. Agenda: Virtual Meetup 6:00 PM - Kickoff Welcome remarks &

    agenda by Jules Damji 6:05 PM - “Year in Review: State of Ray in 2021” by Robert Nishihara 6:15 PM - “What’s New in Ray 1.9 and Beyond” by Zhe Zhang 6:35 PM - “Unifying Preprocessing and Training at Scale with Ray Datasets" by Alex Wu & Clark Zinzow 7:15 PM - "DeltaCAT: A Scalable Data Catalog for Ray Datasets" by Patrick Ames, Amazon
  2. Native Libraries 3rd Party Libraries Your app here! Ecosystem -

    the big picture! Universal framework for distributed computing Run anywhere Library + app ecosystem
  3. “Ray will play an increasingly important role in bringing much

    needed common infrastructure and standardization to the production machine learning ecosystem, both within Uber and the industry at large.” Horovod and more ...
  4. What’s New in Ray 1.9 & Beyond Zhe Zhang Head

    of Open Source Engineering @Anyscale
  5. Overall Themes Core: Reliable and stable at large scale Libraries:

    Easy-to-use high level libraries for production workloads Deployment: Simple and clear paths for deploying clusters and code
  6. Ray Core – Stable at Large Scale Pluggable GCS backend

    Redis GCS Raylet Worker Fetch GCS location via Redis Redis Pubsub KV via GCS Redis backs GCS KV and internal data tables
  7. Ray Core – Stable at Large Scale - Object lifetime

    mgmt - PB level data processing - Rock solid scheduler https://github.com/orgs/ray-project/projects/10/views/5
  8. Libraries: Easy / Production High level plan Dataset Workflow Train

    Serve Pipelines 1.10 Beta Alpha Beta Experimental 1.11 / 1.12 GA Beta (❤ your input!) Alpha/Beta
  9. Deployment – Clear and Simple Ray Client: Interactive development Job

    Submission: Production jobs runtime_env: Dependencies and package
  10. Start learning Ray and contributing … Getting Started: Documentation (docs.ray.io)

    Quick start example, reference guides, etc Join Ray Meetup Revive in Jan 2022. Publish recording to the members https://www.meetup.com/Bay-Area-Ray-Meetup/ Forums (discuss.ray.io) Learn / share with broader Ray community, including core team Ray Slack Connect with the Ray team and community Social Media (@raydistrtibuted, @anyscalecompute) Follow us on Twitter and linkedIn GitHub Check out sources, file an issue, become a contributor, give us a Star :) https://github.com/ray-project/ray