Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
KubeflowではじめるML on Kubernetes #1 Kubeflowの概要とセッ...
Search
Keita Watanabe
August 17, 2022
1
470
KubeflowではじめるML on Kubernetes #1 Kubeflowの概要とセットアップ
Keita Watanabe
August 17, 2022
Tweet
Share
More Decks by Keita Watanabe
See All by Keita Watanabe
Scalable Infrastructure for Large-Scale AI Training with AWS Sagemaker Hyperpod @ Singapore AI Hour
keitaw
0
17
[AWS Summit Japan 2025] Optimizing Foundation Model Development with Amazon SageMaker HyperPod: Insights from Training the Amazon Nova Model
keitaw
0
22
Building foundation models on AWS
keitaw
0
320
[re:Invent2024 Chalktalk] Cost-effectively deploy PyTorch LLMs on AWS Inferentia using Amazon EKS
keitaw
0
90
AWS Summit New York 2024: CMP 301 Demystifying the ML software stack on Amazon EC2 accelerated instances
keitaw
0
380
re:Invent 2023 CMP319 Deploy LLMs with AWS Inferentia & Ray to optimize performance and cost
keitaw
0
37
re:Invent 2023: CMP332 De-mystifying ML software stack on Amazon EC2 accelerated instances
keitaw
1
170
AWS における LLM・GenAI 大規模学習への取り組み / Large scale GenAI・LLM training on AWS
keitaw
1
540
Amazon EC2 シリコン革命 / Amazon EC2 Silicon Innovation
keitaw
0
180
Featured
See All Featured
How STYLIGHT went responsive
nonsquared
100
5.8k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
Scaling GitHub
holman
463
140k
Intergalactic Javascript Robots from Outer Space
tanoku
272
27k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
162
15k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
36
2.5k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.6k
Visualization
eitanlees
148
16k
Designing for Performance
lara
610
69k
The Language of Interfaces
destraynor
161
25k
Transcript
Keita Watanabe Kubernetes Meetup Tokyo # 52 LT KubeflowͰ͡ΊΔ ML
on Kubernetes #1 Kube fl owͷ֓ཁͱηοτΞοϓ
Keita Watanabe Machine Learning Solutions Architect ▶ Self-managedͳMachine LearningͷWorkload Λ୲͍ͯ͠ΔSA
▶ લ৬ͰDatascientist/ML Researcherͱͯ͠ ECαΠτ্ͷը૾ݕࡧػೳͷ։ൃʹैࣄ ▶ Twitter: keitaw09 ▶ Linkedin: keitawatanabe ▶ ݱॴଐ: AWS Japan αϯσΟΤΰͰग़ձͬͨτϦ झຯ
Kubeflowͱ Machine Learning Model-Development Lifecycle (MDLC)ͷ ֤εςοϓʢ+ύΠϓϥΠϯʣΛΧόʔ͢ΔToolͷू·Γ KubeflowΛ࠾༻͢ΔϝϦοτ Composability: ༻్ʹԠͯ͡ඞཁͳComponentsΛΓग़ͯ͠༻͍Δ͜ͱ͕Ͱ͖Δ
Portability: Vendor lock-inΛආ͚Δ͜ͱ͕Ͱ͖Δ Scalability: ෛՙʹԠͯ͡Container/NodeΛεέʔϧ͢Δ͜ͱ͕Ͱ͖Δ
Kubeflow overview Central Dashboard Data Ingestion Data Analysis Data Transform
Data Validation Training AutoML Model eval. and valid. Serving Logging KServe Seldon Kube fl ow Pipelines Kube fl ow Notebooks Katib Training Operators MDLCͷ֤εςοϓ ରԠ͢ΔTool
Kubeflow overview Central Dashboard Data Ingestion Data Analysis Data Transform
Data Validation Training AutoML Model eval. and valid. Serving Logging KServe Seldon Kube fl ow Pipelines Kube fl ow Notebooks Katib Training Operators MDLCͷ֤εςοϓ ରԠ͢ΔTool ࠓΔ͜ͱ: Kube fl owͷηοτΞοϓ
Kubeflow setup Amazon EKS্ͰͷKube fl owڥͷߏங • git • kubectl
• eksctl • kustomize • ࠓճAWS Cloud9্Ͱ࡞ۀ͠·͢ cf. https://www.eksworkshop.com/ 020_prerequisites/workspace/ Prerequisites
Demo
Kubeflow setup
To be
࣍ճ…
ࢀߟࢿྉ • https://www.eksworkshop.com/020_prerequisites/workspace/ • https://www.kube fl ow.org/docs/components/ • https://awslabs.github.io/kube fl
ow-manifests/docs/deployment/vanilla/guide/ • https://www.oreilly.com/library/view/kube fl ow-for-machine/9781492050117/