$30 off During Our Annual Pro Sale. View Details »

Kubeflowではじめる
 ML on Kubernetes #2 Kubeflow Notebooks

Kubeflowではじめる
 ML on Kubernetes #2 Kubeflow Notebooks

Keita Watanabe

October 06, 2022
Tweet

More Decks by Keita Watanabe

Other Decks in Programming

Transcript

  1. Keita Watanabe Kubernetes Meetup Tokyo # 53 LT KubeflowͰ͸͡ΊΔ
 ML

    on Kubernetes #2 Kube fl ow Notebooks
  2. Keita Watanabe Machine Learning Solutions Architect ▶ AWS JapanͰSelf-managedͳMachine Learning


    ͷҊ݅Λ୲౰͍ͯ͠ΔSolutions Architect ▶ લ৬Ͱ͸Datascientist/ML Researcherͱͯ͠
 ECαΠτ্ͷը૾ݕࡧػೳͷ։ൃʹैࣄ ▶ Twitter: keitaw09 ▶ Linkedin: keitawatanabe αϯσΟΤΰͰग़ձͬͨτϦ झຯ
  3. Kubeflow overview Central Dashboard Data Ingestion Data Analysis Data Transform

    Data Validation Training AutoML Model eval. and valid. Serving Logging KServe Seldon Kube fl ow Pipelines Kube fl ow Notebooks Katib Training Operators MDLCͷ֤εςοϓ ରԠ͢ΔTool લճͷ಺༰: Kube fl owͷηοτΞοϓ Slide: https://speakerdeck.com/keitaw/kube fl owdehazimeruml-on-kubernetes-number-1-kube fl owfalsegai-yao-tosetutoatupu
  4. ຊ೔ͷ಺༰: Kubeflow Notebooks

  5. Kubeflow Notebooksͱ͸ • Webϕʔεͷ։ൃ؀ڥΛKubernetes্Ͱఏڙ͢Δπʔϧ • Կ͕خ͍͔͠ • ؅ཧऀɿඞཁͳύοέʔδ͕Πϯετʔϧ͞Εͨ։ൃऀ༻ͷΠϝʔδΛ
 खܰʹఏڙͰ͖Δ +

    RBACʹΑΔnotebook΁ͷΞΫηε੍ޚ • ϢʔβʔɿϩʔΧϧͷܭࢉࢿݯʹཔΔ͜ͱͳ͘ɺ
 Ϋϥελ্ͰNotebook؀ڥ౳Λར༻Ͱ͖Δ
 JupyterLab, RStudio, Visual Studio Code (code-server)Λ
 ωΠςΟϒͰαϙʔτ͍ͯ͠Δ https://www.kube fl ow.org/docs/components/notebooks/overview/
  6. σϑΥϧτͰαϙʔτ͞Ε͍ͯΔΠϝʔδ https://github.com/kube fl ow/kube fl ow/tree/master/components/example-notebook-servers#custom-images • Base: ϕʔεΠϝʔδʢUbuntuʣ •

    PyTorch/TensorFlow Full: 
 
 PyTorch/TensorFlow + Pandas΍ ScipyɺXGBoost౳ͷΑ͘࢖ΘΕΔ ϥΠϒϥϦ • Jupyter imageʹ͸GPU൛΋͋Δ • Custom image: BaseΛجʹ࡞੒
 ʢ࡞੒ํ๏͸ԼهͷURLΛࢀরʣ https://www.kube fl ow.org/docs/components/notebooks/container-images/
  7. Notebookͷ৽ن࡞੒

  8. Notebookͷ৽ن࡞੒

  9. Notebookͷ৽ن࡞੒ ❶ ❷ ❸ ❹

  10. Kubeflow NotebooksͰJupyterLab σʔλͷղੳ΍Ϟσϧͷ։ൃɺ͞Βʹ͸Kube fl ow Pipelinesʢৄࡉ͸ࠓޙͷLTͰʣ Λ༻͍ͨύΠϓϥΠϯͷߏஙͳͲ͕ߦ͑Δ

  11. Kubeflow NotebooksͰVSCode

  12. ࣍ճ͸… Kubeflow Pipelines

  13. ࢀߟࢿྉ • https://www.kube fl ow.org/docs/components/notebooks/overview/ • https://github.com/kube fl ow/kube fl

    ow/tree/master/components/example- notebook-servers#custom-images • https://github.com/kube fl ow/pipelines/blob/sdk/release-1.8/samples/core/ lightweight_component/lightweight_component.ipynb • https://github.com/aws-samples/aws-do-kube fl ow