Slide 1
Slide 1 text
1
Meets MLOps
ML@Loft #9
Lain Matsuoka
@lain_m21
Slide 2
Slide 2 text
2
! ॴଐɿAIνʔϜ ιϑτΣΞΤϯδχΞ
○ 2019/01ʙ
! ϝϧΧϦͰͬͨϓϩδΣΫτʢൈਮʣ
○ Ձ֨αδΣετ
○ τϨʔσΟϯάΧʔυը૾ೝࣝ
○ ݕࡧϦϥϯΩϯάʢόοΫΤϯυʣ
○ ΦϯϥΠϯ࣮ݧڥߏஙʢόοΫΤϯυʣ
! ϞσϦϯά͔ΒόοΫΤϯυɺΠϯϑϥ։ൃ·
Ͱ෯͘୲
Lain Matsuoka/দԬ ྰԻ
Slide 3
Slide 3 text
3
ϝϧΧϦͷػցֶशαʔϏε
Agenda
Istioͱʁ
Istio͕ͨΒ͢Ըܙ
02
03
01
Slide 4
Slide 4 text
4
ϝϧΧϦͷػցֶशαʔϏε
Slide 5
Slide 5 text
5
ݱߦͷػցֶशαʔϏεͨͪ
! ҧग़ݕʢෳυϝΠϯʣ
! Ձ֨αδΣετ
! ײಈग़ʢใͷࣗಈิʣ
! ૹํ๏αδΣετ
! ը૾ݕࡧ
! etc.
Slide 6
Slide 6 text
6
ݱߦͷػցֶशαʔϏεͨͪ
! ҧग़ݕʢෳυϝΠϯʣ = API + TF Serving
! Ձ֨αδΣετ = API + TF Serving
! ײಈग़ʢใͷࣗಈิʣ = API + TF/ONNX Serving + Faiss
! ૹํ๏αδΣετ = API + PostgreSQL
! ը૾ݕࡧ ɹ = API + TF Serving + Faiss
! etc.
ଟ༷ͳΞʔΩςΫνϟͷਪαʔϏεΛKubernetesΫϥελʔ্Ͱӡ༻
Slide 7
Slide 7 text
7
ػցֶशαʔϏεͷಛੑ
! ୯७ʹܭࢉෛՙ͕ߴ͍
! ෳίϯϙʔωϯτΛΈ߹ΘͤͨෳࡶͳΞʔΩςΫνϟʹͳΓ͍͢
○ લॲཧɺޙॲཧΛߦ͏APIαʔόʔ
○ Deep LearningͷਪαʔόʔʢTF Serving, ONNXRuntimeʣ
○ ྨࣅݕࡧαʔόʔʢFaiss, NMSLIBʣ
○ छʑͷσʔλϕʔεʢRedis, PostgreSQLʣ
IstioͰαʔϏεϝογϡΛߏங͠ɺޮతʹཧ
Slide 8
Slide 8 text
8
ͱʁ
Slide 9
Slide 9 text
9
ϚΠΫϩαʔϏεͱαʔϏεϝογϡ
Control Plane
Data Plane
Service A
Service
Container
Envoy
Proxy
Service B
Service
Container
Envoy
Proxy
Service C
Service
Container
Envoy
Proxy
Service D
Service
Container
Envoy
Proxy
! ϚΠΫϩαʔϏε܈ͷӡ༻ͱ࿈ܞΛޮԽ͢Δ
Slide 10
Slide 10 text
10
Istioͷओͳػೳ
! ΞϓϦέʔγϣϯΞάϊεςΟοΫʹɺԼهͷػೳ͕ఏڙ͞ΕΔ
○ τϥϑΟοΫ੍ޚ
○ ௨৴༰ϝτϦΫεͷऔಘɺࢹ
○ ೝূɺΞΫηε੍ݶɺ҉߸Խ
Slide 11
Slide 11 text
11
Istioͷओͳػೳ
! ΞϓϦέʔγϣϯΞάϊεςΟοΫʹɺԼهͷػೳ͕ఏڙ͞ΕΔ
○ τϥϑΟοΫ੍ޚ
○ ௨৴༰ϝτϦΫεͷऔಘɺࢹ
○ ೝূɺΞΫηε੍ݶɺ҉߸Խ
Slide 12
Slide 12 text
12
͕ͨΒ͢Ըܙ
Slide 13
Slide 13 text
13
ෛՙϨΠςϯγͷϞχλϦϯάɺࣗಈεέʔϦϯά
! Envoy͕औಘ͢ΔϝτϦΫεΛPrometheus + GrafanaͰࢹ
○ ඵؒϦΫΤετ
○ ϨΠςϯγ
○ CPUɾϝϞϦར༻
! औಘͨ͠ϝτϦΫεΛݩʹɺHPAʢHorizontal Pod AutoscalerʣͰࣗಈε
έʔϦϯά
Slide 14
Slide 14 text
14
ΧφϦΞσϓϩΠʹΑΔθϩμϯλΠϜϦϦʔε
! IstioͷVirtual ServiceʹΑΔτϥϑΟοΫ੍ޚΛ༻
○ KubernetesͷServiceͷΈͰෆՄೳͳཻͰ੍ޚՄೳ
! ࠶ֶशͨ͠Ϟσϧɺ৽نϑΟʔνϟʔΛ҆શʹϦϦʔεͰ͖Δ
○ ྲྀྔΛ0ʹ͢Δ͚ͩͰ҆શʹϩʔϧόοΫ
- route:
- destination:
host: service-a.default.svc.cluster.local
weight: 99
- destination:
host: service-b.default.svc.cluster.local
weight: 1
Service A
Service B
Virtual
Service
Service A
Service B
Virtual
Service
Slide 15
Slide 15 text
15
ϛϥʔϦϯάʹΑΔຊ൪ڥͰͷαʔϏεݕূ
! IstioͷVirtual ServiceϛϥʔϦϯάػೳαϙʔτ
○ ΦϦδφϧͷτϥϑΟοΫͷશͳίϐʔΛྲྀೖͤ͞ΒΕΔ
! ࣮ڥͰͷϦΫΤετͰ҆શʹಈ࡞ݕূͰ͖Δ
Service A
Service B
Virtual
Service
- route:
- destination:
host: service-a.default.svc.cluster.local
weight: 100
mirror:
host: service-b.default.svc.cluster.local
mirror_percent: 100
Slide 16
Slide 16 text
16
·ͱΊ
Slide 17
Slide 17 text
17
IstioͰػցֶशαʔϏεͷӡ༻Λ͍͍ײ͡ʹʂʂ
! ػցֶशαʔϏεෳίϯϙʔωϯτ͔ΒͳΔෳࡶ͔ͭߴෛՙͳαʔϏ
εʹͳΓ͕ͪ
! IstioʹΑΔαʔϏεϝογϡΛར༻͢Δ͜ͱͰɺޮΑ͘ෳͷػցֶश
αʔϏεΛӡ༻
○ ΞϓϦέʔγϣϯଆʹखΛೖΕͳͯ͘Α͍
○ ࢹͦΕΛϕʔεʹͨࣗ͠ಈεέʔϦϯά
○ τϥϑΟοΫ੍ޚΛར༻ͨ͠ݕূɺ҆શͳϦϦʔεɾϩʔϧόοΫ