Tweet
Tweet

Slide 1

Slide 1 text

Powering ODH With Ray Erik Erlandson, Red Hat, Inc. [email protected] @ManyAngled

Slide 2

Slide 2 text

Or... Erik Erlandson, Red Hat, Inc. [email protected] @ManyAngled

Slide 3

Slide 3 text

Jupyter & Ray In The Cloud Erik Erlandson, Red Hat, Inc. [email protected] @ManyAngled

Slide 4

Slide 4 text

Landscape Motivations Open Data Hub and Jupyter in Context Ray on ODH Demo Community Collaborations

Slide 5

Slide 5 text

Native Ray Libraries ● Tune: Scalable Hyperparameter Tuning ● RLlib: Scalable Reinforcement Learning ● RaySGD: Distributed Training Wrappers ● Ray Serve: Scalable and Programmable Serving

Slide 6

Slide 6 text

Ray Community Integrations ● XGBoost ● Dask ● Horovod ● sklearn ● Spacy ● huggingface https://docs.ray.io/en/master/ray-libraries.html

Slide 7

Slide 7 text

Ray Community Integrations ● XGBoost ● Dask ● Horovod ● sklearn ● Spacy ● huggingface https://docs.ray.io/en/master/ray-libraries.html

Slide 8

Slide 8 text

Literate And Interactive Ray... https://docs.ray.io/en/master/ray-libraries.html

Slide 9

Slide 9 text

Hosted In The Cloud https://docs.ray.io/en/master/ray-libraries.html

Slide 10

Slide 10 text

Jupyter + Ray 1.X Jupyter + Ray Head Pod Ray Worker Pods

Slide 11

Slide 11 text

Jupyter + Ray 1.X Jupyter + Ray Head Pod Ray Worker Pods

Slide 12

Slide 12 text

Jupyter + Ray 2.0 Ray Worker Pods Ray Head Pod Jupyter Pod

Slide 13

Slide 13 text

Jupyter ...

Slide 14

Slide 14 text

Jupyter via Open Data Hub

Slide 15

Slide 15 text

Open Data Hub Is ... Open Source Downstream Reference Platform Federated Meta Operator

Slide 16

Slide 16 text

Open Data Hub Is ... Open Source Downstream Reference Platform Federated Meta Operator

Slide 17

Slide 17 text

Open Data Hub Is ... Open Source Downstream Reference Platform Federated Meta Operator

Slide 18

Slide 18 text

Open Data Hub Is ... Open Source Downstream Reference Platform Federated Meta Operator

Slide 19

Slide 19 text

Open Data Hub Is ... Open Source Downstream Reference Platform Federated Meta Operator

Slide 20

Slide 20 text

Data Science with ODH Set goals Gather and prepare data Develop ML model Deploy ML models in app dev process Implement Apps & Inference ML models Monitoring & Management

Slide 21

Slide 21 text

Data Science with ODH Set goals Gather and prepare data Develop ML model Deploy ML models in app dev process Implement Apps & Inference ML models Monitoring & Management

Slide 22

Slide 22 text

Data Science with ODH Set goals Gather and prepare data Develop ML model Deploy ML models in app dev process Implement Apps & Inference ML models Monitoring & Management App developer IT operations Data engineer Business leadership Data scientists ML Engineer

Slide 23

Slide 23 text

Data Science with ODH Set goals Gather and prepare data Develop ML model Deploy ML models in app dev process Implement Apps & Inference ML models Monitoring & Management App developer IT operations Data engineer Business leadership Data scientists ML Engineer Seldon Jupyter Ceph Spark TensorFlow Kafka SuperSet Argo/Airflow/Tekton Hue Prometheus/Grafana Argo/Airflow/Tekton Ceph Kafka Seldon Middleware M odel to M icroservice

Slide 24

Slide 24 text

Dog-Fooding ODH at Red Hat Application Logs Applications in the product release pipeline store their runtime logs in our system. These groups are also engaged for anomaly detection Cluster Metrics Operational metrics from OpenShift clusters. AIOps is engaged here. Customer Support Data Storage of customer data like SOSReports, customer feedback, etc.

Slide 25

Slide 25 text

Analogy: Spark on ODH ODH JupyterHub Launcher

Slide 26

Slide 26 text

Analogy: Spark on ODH ODH JupyterHub Launcher Jupyter Environment

Slide 27

Slide 27 text

Analogy: Spark on ODH ODH JupyterHub Launcher Spark SingleUser Profile Jupyter Environment

Slide 28

Slide 28 text

Analogy: Spark on ODH ODH JupyterHub Launcher Spark SingleUser Profile Spark Cluster Service Template Jupyter Environment

Slide 29

Slide 29 text

Analogy: Spark on ODH Spark cluster ODH JupyterHub Launcher Spark SingleUser Profile Spark Cluster Service Template Jupyter Environment

Slide 30

Slide 30 text

Analogy: Spark on ODH Spark cluster ODH JupyterHub Launcher Spark SingleUser Profile Spark Cluster Service Template Jupyter Environment

Slide 31

Slide 31 text

Analogy: Spark on ODH Spark cluster Spark SingleUser Profile Spark Cluster Service Template ConfigMap ConfigMap

Slide 32

Slide 32 text

Ray on ODH? Ray cluster ODH JupyterHub Launcher Ray SingleUser Profile Ray Cluster Service Template Jupyter Environment

Slide 33

Slide 33 text

Ray Single User Profile

Slide 34

Slide 34 text

Ray Cluster Service Template

Slide 35

Slide 35 text

Demo: Ray on ODH! Ray cluster ODH JupyterHub Launcher Ray SingleUser Profile Ray Cluster Service Template Jupyter Environment

Slide 36

Slide 36 text

Ray on ODH at the Mass-Open Cloud Led by Boston University, the MOC is a collaborative effort among BU, Harvard, UMass Amherst, MIT, and Northeastern University, as well as the Massachusetts Green High-Performance Computing Center (MGHPCC) and Oak Ridge National Laboratory (ORNL). It is supported by a broad alliance of industry partners, including Red Hat.

Slide 37

Slide 37 text

Ray on MOC ● Maximum 5 workers + 1 head ● 1 CPU, 1 GB memory ● Pre-installed:

Slide 38

Slide 38 text

Operate First https://www.operate-first.cloud/ Developing Software In The Open Operating Software and Services In the Open

Slide 39

Slide 39 text

Operate First PRs for Ray https://github.com/operate-first/support/issues/102

Slide 40

Slide 40 text

Collaboration: IBM ● Ray with Code Engine ● Ray on IBM OpenShift Clusters ● Scikit-Learn pipelines on Ray ● Ray Use Cases ○ Machine Learning Model Explorations ○ Earth Science

Slide 41

Slide 41 text

IBM Research at Ray Summit Raghu Ganti: Scaling and Unifying SciKit Learn and Spark Pipelines using Ray Linsong Chu: Serverless Earth Science Data Labeling using Unsupervised Deep Learning with Ray

Slide 42

Slide 42 text

Roadmap ● Community Ray Operator in Catalog ● Maintain Ray Images via Project Thoth ● Community Use Cases With Jupyter ● Formal Integration With KF and ODH ● KF Pipeline Nodes Backed by Ray

Slide 43

Slide 43 text

Call To Action ● Play with Ray on Jupyter up on MOC ● File issues and PRs with op-1st ● Report Back! [email protected] https://www.operate-first.cloud/users/moc-ray-demo/README.md https://odh.operate-first.cloud/

Slide 44

Slide 44 text

1 1 π ≈ 4 Σ Σ( + )