Istio Meets ML Ops - ML@Loft#9

Transcript

1. 1 Meets MLOps ML@Loft #9 Lain Matsuoka @lain_m21
2. 2 ! ॴଐɿAIνʔϜ ιϑτ΢ΣΞΤϯδχΞ ◦ 2019/01ʙ ! ϝϧΧϦͰ΍ͬͨϓϩδΣΫτʢൈਮʣ ◦ Ձ֨αδΣετ ◦ τϨʔσΟϯάΧʔυը૾ೝࣝ ◦ ݕࡧϦϥϯΩϯάʢόοΫΤϯυʣ ◦ ΦϯϥΠϯ࣮ݧ؀ڥߏஙʢόοΫΤϯυʣ ! ϞσϦϯά͔ΒόοΫΤϯυɺΠϯϑϥ։ൃ· Ͱ෯޿͘୲౰ Lain Matsuoka/দԬ ྰԻ
3. 3 ϝϧΧϦͷػցֶशαʔϏε Agenda Istioͱ͸ʁ Istio͕΋ͨΒ͢Ըܙ 02 03 01
4. 4 ϝϧΧϦͷػցֶशαʔϏε
5. 5 ݱߦͷػցֶशαʔϏεͨͪ ! ҧ൓ग़඼ݕ஌ʢෳ਺υϝΠϯʣ ! Ձ֨αδΣετ ! ײಈग़඼ʢ৘ใͷࣗಈิ׬ʣ ! ഑ૹํ๏αδΣετ ! ը૾ݕࡧ ! etc.
6. 6 ݱߦͷػցֶशαʔϏεͨͪ ! ҧ൓ग़඼ݕ஌ʢෳ਺υϝΠϯʣ = API + TF Serving ! Ձ֨αδΣετ = API + TF Serving ! ײಈग़඼ʢ৘ใͷࣗಈิ׬ʣ = API + TF/ONNX Serving + Faiss ! ഑ૹํ๏αδΣετ = API + PostgreSQL ! ը૾ݕࡧ ɹ = API + TF Serving + Faiss ! etc. ଟ༷ͳΞʔΩςΫνϟͷਪ࿦αʔϏεΛKubernetesΫϥελʔ্Ͱӡ༻
7. 7 ػցֶशαʔϏεͷಛੑ ! ୯७ʹܭࢉෛՙ͕ߴ͍ ! ෳ਺ίϯϙʔωϯτΛ૊Έ߹ΘͤͨෳࡶͳΞʔΩςΫνϟʹͳΓ΍͍͢ ◦ લॲཧɺޙॲཧΛߦ͏APIαʔόʔ ◦ Deep Learningͷਪ࿦αʔόʔʢTF Serving, ONNXRuntimeʣ ◦ ྨࣅ౓ݕࡧαʔόʔʢFaiss, NMSLIBʣ ◦ छʑͷσʔλϕʔεʢRedis, PostgreSQLʣ IstioͰαʔϏεϝογϡΛߏங͠ɺޮ཰తʹ؅ཧ
8. 8 ͱ͸ʁ
9. 9 ϚΠΫϩαʔϏεͱαʔϏεϝογϡ Control Plane Data Plane Service A Service Container Envoy Proxy Service B Service Container Envoy Proxy Service C Service Container Envoy Proxy Service D Service Container Envoy Proxy ! ϚΠΫϩαʔϏε܈ͷӡ༻ͱ࿈ܞΛޮ཰Խ͢Δ
10. 10 Istioͷओͳػೳ ! ΞϓϦέʔγϣϯΞάϊεςΟοΫʹɺԼهͷػೳ͕ఏڙ͞ΕΔ ◦ τϥϑΟοΫ੍ޚ ◦ ௨৴಺༰΍ϝτϦΫεͷऔಘɺ؂ࢹ ◦ ೝূɺΞΫηε੍ݶɺ҉߸Խ
11. 11 Istioͷओͳػೳ ! ΞϓϦέʔγϣϯΞάϊεςΟοΫʹɺԼهͷػೳ͕ఏڙ͞ΕΔ ◦ τϥϑΟοΫ੍ޚ ◦ ௨৴಺༰΍ϝτϦΫεͷऔಘɺ؂ࢹ ◦ ೝূɺΞΫηε੍ݶɺ҉߸Խ
12. 12 ͕΋ͨΒ͢Ըܙ
13. 13 ෛՙ΍ϨΠςϯγͷϞχλϦϯάɺࣗಈεέʔϦϯά ! Envoy͕औಘ͢ΔϝτϦΫεΛPrometheus + GrafanaͰ؂ࢹ ◦ ඵؒϦΫΤετ਺ ◦ ϨΠςϯγ ◦ CPUɾϝϞϦར༻཰ ! औಘͨ͠ϝτϦΫεΛݩʹɺHPAʢHorizontal Pod AutoscalerʣͰࣗಈε έʔϦϯά
14. 14 ΧφϦΞσϓϩΠʹΑΔθϩμ΢ϯλΠϜϦϦʔε ! IstioͷVirtual ServiceʹΑΔτϥϑΟοΫ੍ޚΛ࢖༻ ◦ KubernetesͷServiceͷΈͰ͸ෆՄೳͳཻ౓Ͱ੍ޚՄೳ ! ࠶ֶशͨ͠Ϟσϧ΍ɺ৽نϑΟʔνϟʔΛ҆શʹϦϦʔεͰ͖Δ ◦ ྲྀྔΛ0ʹ͢Δ͚ͩͰ҆શʹϩʔϧόοΫ - route: - destination: host: service-a.default.svc.cluster.local
 weight: 99 - destination: host: service-b.default.svc.cluster.local
 weight: 1 Service A Service B Virtual Service Service A Service B Virtual Service
15. 15 ϛϥʔϦϯάʹΑΔຊ൪؀ڥͰͷαʔϏεݕূ ! IstioͷVirtual Service͸ϛϥʔϦϯάػೳ΋αϙʔτ ◦ ΦϦδφϧͷτϥϑΟοΫͷ׬શͳίϐʔΛྲྀೖͤ͞ΒΕΔ ! ࣮؀ڥͰͷϦΫΤετͰ҆શʹಈ࡞ݕূͰ͖Δ Service A Service B Virtual Service - route: - destination: host: service-a.default.svc.cluster.local
 weight: 100 mirror: host: service-b.default.svc.cluster.local
 mirror_percent: 100
16. 16 ·ͱΊ
17. 17 IstioͰػցֶशαʔϏεͷӡ༻Λ͍͍ײ͡ʹʂʂ ! ػցֶशαʔϏε͸ෳ਺ίϯϙʔωϯτ͔ΒͳΔෳࡶ͔ͭߴෛՙͳαʔϏ εʹͳΓ͕ͪ ! IstioʹΑΔαʔϏεϝογϡΛར༻͢Δ͜ͱͰɺޮ཰Α͘ෳ਺ͷػցֶश αʔϏεΛӡ༻ ◦ ΞϓϦέʔγϣϯଆʹखΛೖΕͳͯ͘Α͍ ◦ ؂ࢹ΍ͦΕΛϕʔεʹͨࣗ͠ಈεέʔϦϯά ◦ τϥϑΟοΫ੍ޚΛར༻ͨ͠ݕূɺ҆શͳϦϦʔεɾϩʔϧόοΫ