Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
KubeflowではじめるML on Kubernetes #1 Kubeflowの概要とセッ...
Search
Keita Watanabe
August 17, 2022
1
460
KubeflowではじめるML on Kubernetes #1 Kubeflowの概要とセットアップ
Keita Watanabe
August 17, 2022
Tweet
Share
More Decks by Keita Watanabe
See All by Keita Watanabe
[AWS Summit Japan 2025] Optimizing Foundation Model Development with Amazon SageMaker HyperPod: Insights from Training the Amazon Nova Model
keitaw
0
4
Building foundation models on AWS
keitaw
0
150
[re:Invent2024 Chalktalk] Cost-effectively deploy PyTorch LLMs on AWS Inferentia using Amazon EKS
keitaw
0
66
AWS Summit New York 2024: CMP 301 Demystifying the ML software stack on Amazon EC2 accelerated instances
keitaw
0
310
re:Invent 2023 CMP319 Deploy LLMs with AWS Inferentia & Ray to optimize performance and cost
keitaw
0
32
re:Invent 2023: CMP332 De-mystifying ML software stack on Amazon EC2 accelerated instances
keitaw
0
140
AWS における LLM・GenAI 大規模学習への取り組み / Large scale GenAI・LLM training on AWS
keitaw
1
530
Amazon EC2 シリコン革命 / Amazon EC2 Silicon Innovation
keitaw
0
180
Introduction to Kserve
keitaw
0
590
Featured
See All Featured
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.4k
Done Done
chrislema
184
16k
Fireside Chat
paigeccino
37
3.5k
Navigating Team Friction
lara
187
15k
Raft: Consensus for Rubyists
vanstee
140
7k
Agile that works and the tools we love
rasmusluckow
329
21k
Making the Leap to Tech Lead
cromwellryan
134
9.3k
Adopting Sorbet at Scale
ufuk
77
9.4k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
657
60k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Speed Design
sergeychernyshev
32
1k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Transcript
Keita Watanabe Kubernetes Meetup Tokyo # 52 LT KubeflowͰ͡ΊΔ ML
on Kubernetes #1 Kube fl owͷ֓ཁͱηοτΞοϓ
Keita Watanabe Machine Learning Solutions Architect ▶ Self-managedͳMachine LearningͷWorkload Λ୲͍ͯ͠ΔSA
▶ લ৬ͰDatascientist/ML Researcherͱͯ͠ ECαΠτ্ͷը૾ݕࡧػೳͷ։ൃʹैࣄ ▶ Twitter: keitaw09 ▶ Linkedin: keitawatanabe ▶ ݱॴଐ: AWS Japan αϯσΟΤΰͰग़ձͬͨτϦ झຯ
Kubeflowͱ Machine Learning Model-Development Lifecycle (MDLC)ͷ ֤εςοϓʢ+ύΠϓϥΠϯʣΛΧόʔ͢ΔToolͷू·Γ KubeflowΛ࠾༻͢ΔϝϦοτ Composability: ༻్ʹԠͯ͡ඞཁͳComponentsΛΓग़ͯ͠༻͍Δ͜ͱ͕Ͱ͖Δ
Portability: Vendor lock-inΛආ͚Δ͜ͱ͕Ͱ͖Δ Scalability: ෛՙʹԠͯ͡Container/NodeΛεέʔϧ͢Δ͜ͱ͕Ͱ͖Δ
Kubeflow overview Central Dashboard Data Ingestion Data Analysis Data Transform
Data Validation Training AutoML Model eval. and valid. Serving Logging KServe Seldon Kube fl ow Pipelines Kube fl ow Notebooks Katib Training Operators MDLCͷ֤εςοϓ ରԠ͢ΔTool
Kubeflow overview Central Dashboard Data Ingestion Data Analysis Data Transform
Data Validation Training AutoML Model eval. and valid. Serving Logging KServe Seldon Kube fl ow Pipelines Kube fl ow Notebooks Katib Training Operators MDLCͷ֤εςοϓ ରԠ͢ΔTool ࠓΔ͜ͱ: Kube fl owͷηοτΞοϓ
Kubeflow setup Amazon EKS্ͰͷKube fl owڥͷߏங • git • kubectl
• eksctl • kustomize • ࠓճAWS Cloud9্Ͱ࡞ۀ͠·͢ cf. https://www.eksworkshop.com/ 020_prerequisites/workspace/ Prerequisites
Demo
Kubeflow setup
To be
࣍ճ…
ࢀߟࢿྉ • https://www.eksworkshop.com/020_prerequisites/workspace/ • https://www.kube fl ow.org/docs/components/ • https://awslabs.github.io/kube fl
ow-manifests/docs/deployment/vanilla/guide/ • https://www.oreilly.com/library/view/kube fl ow-for-machine/9781492050117/