Kubeflowではじめる ML on Kubernetes #2 Kubeflow Notebooks

https://k8sjp.connpass.com/event/259350/ 発表資料

Transcript

1. Keita Watanabe Kubernetes Meetup Tokyo # 53 LT
   KubeflowͰ͸͡ΊΔ

   ML on Kubernetes
   #2 Kube
   fl
   ow Notebooks
   

   View Slide

2. Keita Watanabe
   Machine Learning Solutions Architect
   ▶ AWS JapanͰSelf-managedͳMachine Learning

   ͷҊ݅Λ୲౰͍ͯ͠ΔSolutions Architect
   
   ▶ લ৬Ͱ͸Datascientist/ML Researcherͱͯ͠

   ECαΠτ্ͷը૾ݕࡧػೳͷ։ൃʹैࣄ
   
   ▶ Twitter: keitaw09
   
   ▶ Linkedin: keitawatanabe
   
   αϯσΟΤΰͰग़ձͬͨτϦ
   झຯ
   

   View Slide

3. Kubeflow overview
   Central Dashboard
   Data
   
   
   Ingestion
   Data
   
   
   Analysis
   Data
   Transform
   Data
   
   
   Validation
   Training
   AutoML
   Model eval.
   and valid. Serving
   Logging
   KServe
   Seldon
   Kube
   fl
   ow Pipelines
   Kube
   fl
   ow Notebooks Katib
   Training Operators
   MDLCͷ֤εςοϓ
   ରԠ͢ΔTool
   લճͷ಺༰: Kube
   fl
   owͷηοτΞοϓ
   Slide: https://speakerdeck.com/keitaw/kube
   fl
   owdehazimeruml-on-kubernetes-number-1-kube
   fl
   owfalsegai-yao-tosetutoatupu
   

   View Slide

4. ຊ೔ͷ಺༰: Kubeflow Notebooks
   

   View Slide

5. Kubeflow Notebooksͱ͸
   • Webϕʔεͷ։ൃ؀ڥΛKubernetes্Ͱఏڙ͢Δπʔϧ
   
   • Կ͕خ͍͔͠
   
   • ؅ཧऀɿඞཁͳύοέʔδ͕Πϯετʔϧ͞Εͨ։ൃऀ༻ͷΠϝʔδΛ

   खܰʹఏڙͰ͖Δ + RBACʹΑΔnotebook΁ͷΞΫηε੍ޚ
   
   • ϢʔβʔɿϩʔΧϧͷܭࢉࢿݯʹཔΔ͜ͱͳ͘ɺ

   Ϋϥελ্ͰNotebook؀ڥ౳Λར༻Ͱ͖Δ

   JupyterLab, RStudio, Visual Studio Code (code-server)Λ

   ωΠςΟϒͰαϙʔτ͍ͯ͠Δ
   https://www.kube
   fl
   ow.org/docs/components/notebooks/overview/
   

   View Slide

6. σϑΥϧτͰαϙʔτ͞Ε͍ͯΔΠϝʔδ
   https://github.com/kube
   fl
   ow/kube
   fl
   ow/tree/master/components/example-notebook-servers#custom-images
   • Base: ϕʔεΠϝʔδʢUbuntuʣ
   
   • PyTorch/TensorFlow Full: 

   

   PyTorch/TensorFlow + Pandas΍
   ScipyɺXGBoost౳ͷΑ͘࢖ΘΕΔ
   ϥΠϒϥϦ
   
   • Jupyter imageʹ͸GPU൛΋͋Δ
   
   • Custom image: BaseΛجʹ࡞੒

   ʢ࡞੒ํ๏͸ԼهͷURLΛࢀরʣ
   https://www.kube
   fl
   ow.org/docs/components/notebooks/container-images/
   

   View Slide

7. Notebookͷ৽ن࡞੒
   

   View Slide

8. Notebookͷ৽ن࡞੒
   

   View Slide

9. Notebookͷ৽ن࡞੒
   ❶
   ❷
   ❸
   ❹
   

   View Slide

10. Kubeflow NotebooksͰJupyterLab
    σʔλͷղੳ΍Ϟσϧͷ։ൃɺ͞Βʹ͸Kube
    fl
    ow Pipelinesʢৄࡉ͸ࠓޙͷLTͰʣ
    Λ༻͍ͨύΠϓϥΠϯͷߏஙͳͲ͕ߦ͑Δ
    

    View Slide

11. Kubeflow NotebooksͰVSCode
    

    View Slide

12. ࣍ճ͸… Kubeflow Pipelines
    

    View Slide

13. ࢀߟࢿྉ
    • https://www.kube
    fl
    ow.org/docs/components/notebooks/overview/
    
    • https://github.com/kube
    fl
    ow/kube
    fl
    ow/tree/master/components/example-
    notebook-servers#custom-images
    
    • https://github.com/kube
    fl
    ow/pipelines/blob/sdk/release-1.8/samples/core/
    lightweight_component/lightweight_component.ipynb
    
    • https://github.com/aws-samples/aws-do-kube
    fl
    ow
    

    View Slide