Kubeflowではじめる ML on Kubernetes #2 Kubeflow Notebooks

https://k8sjp.connpass.com/event/259350/ 発表資料

Transcript

1. Keita Watanabe Kubernetes Meetup Tokyo # 53 LT KubeflowͰ͸͡ΊΔ
 ML on Kubernetes #2 Kube fl ow Notebooks
2. Keita Watanabe Machine Learning Solutions Architect ▶ AWS JapanͰSelf-managedͳMachine Learning
 ͷҊ݅Λ୲౰͍ͯ͠ΔSolutions Architect ▶ લ৬Ͱ͸Datascientist/ML Researcherͱͯ͠
 ECαΠτ্ͷը૾ݕࡧػೳͷ։ൃʹैࣄ ▶ Twitter: keitaw09 ▶ Linkedin: keitawatanabe αϯσΟΤΰͰग़ձͬͨτϦ झຯ
3. Kubeflow overview Central Dashboard Data Ingestion Data Analysis Data Transform Data Validation Training AutoML Model eval. and valid. Serving Logging KServe Seldon Kube fl ow Pipelines Kube fl ow Notebooks Katib Training Operators MDLCͷ֤εςοϓ ରԠ͢ΔTool લճͷ಺༰: Kube fl owͷηοτΞοϓ Slide: https://speakerdeck.com/keitaw/kube fl owdehazimeruml-on-kubernetes-number-1-kube fl owfalsegai-yao-tosetutoatupu
4. ຊ೔ͷ಺༰: Kubeflow Notebooks
5. Kubeflow Notebooksͱ͸ • Webϕʔεͷ։ൃ؀ڥΛKubernetes্Ͱఏڙ͢Δπʔϧ • Կ͕خ͍͔͠ • ؅ཧऀɿඞཁͳύοέʔδ͕Πϯετʔϧ͞Εͨ։ൃऀ༻ͷΠϝʔδΛ
 खܰʹఏڙͰ͖Δ + RBACʹΑΔnotebook΁ͷΞΫηε੍ޚ • ϢʔβʔɿϩʔΧϧͷܭࢉࢿݯʹཔΔ͜ͱͳ͘ɺ
 Ϋϥελ্ͰNotebook؀ڥ౳Λར༻Ͱ͖Δ
 JupyterLab, RStudio, Visual Studio Code (code-server)Λ
 ωΠςΟϒͰαϙʔτ͍ͯ͠Δ https://www.kube fl ow.org/docs/components/notebooks/overview/
6. σϑΥϧτͰαϙʔτ͞Ε͍ͯΔΠϝʔδ https://github.com/kube fl ow/kube fl ow/tree/master/components/example-notebook-servers#custom-images • Base: ϕʔεΠϝʔδʢUbuntuʣ • PyTorch/TensorFlow Full: 
 
 PyTorch/TensorFlow + Pandas΍ ScipyɺXGBoost౳ͷΑ͘࢖ΘΕΔ ϥΠϒϥϦ • Jupyter imageʹ͸GPU൛΋͋Δ • Custom image: BaseΛجʹ࡞੒
 ʢ࡞੒ํ๏͸ԼهͷURLΛࢀরʣ https://www.kube fl ow.org/docs/components/notebooks/container-images/
7. Notebookͷ৽ن࡞੒
8. Notebookͷ৽ن࡞੒
9. Notebookͷ৽ن࡞੒ ❶ ❷ ❸ ❹
10. Kubeflow NotebooksͰJupyterLab σʔλͷղੳ΍Ϟσϧͷ։ൃɺ͞Βʹ͸Kube fl ow Pipelinesʢৄࡉ͸ࠓޙͷLTͰʣ Λ༻͍ͨύΠϓϥΠϯͷߏஙͳͲ͕ߦ͑Δ
11. Kubeflow NotebooksͰVSCode
12. ࣍ճ͸… Kubeflow Pipelines
13. ࢀߟࢿྉ • https://www.kube fl ow.org/docs/components/notebooks/overview/ • https://github.com/kube fl ow/kube fl ow/tree/master/components/example- notebook-servers#custom-images • https://github.com/kube fl ow/pipelines/blob/sdk/release-1.8/samples/core/ lightweight_component/lightweight_component.ipynb • https://github.com/aws-samples/aws-do-kube fl ow