Istio Meets ML Ops - ML@Loft#9

Istio Meets ML Ops - ML@Loft#9

Cc49268bebed3e30cb943a9de7b6eb3a?s=128

Lain Matsuoka

December 19, 2019
Tweet

Transcript

  1. 1 Meets MLOps ML@Loft #9 Lain Matsuoka @lain_m21

  2. 2 ! ॴଐɿAIνʔϜ ιϑτ΢ΣΞΤϯδχΞ ◦ 2019/01ʙ ! ϝϧΧϦͰ΍ͬͨϓϩδΣΫτʢൈਮʣ ◦ Ձ֨αδΣετ

    ◦ τϨʔσΟϯάΧʔυը૾ೝࣝ ◦ ݕࡧϦϥϯΩϯάʢόοΫΤϯυʣ ◦ ΦϯϥΠϯ࣮ݧ؀ڥߏஙʢόοΫΤϯυʣ ! ϞσϦϯά͔ΒόοΫΤϯυɺΠϯϑϥ։ൃ· Ͱ෯޿͘୲౰ Lain Matsuoka/দԬ ྰԻ
  3. 3 ϝϧΧϦͷػցֶशαʔϏε Agenda Istioͱ͸ʁ Istio͕΋ͨΒ͢Ըܙ 02 03 01

  4. 4 ϝϧΧϦͷػցֶशαʔϏε

  5. 5 ݱߦͷػցֶशαʔϏεͨͪ ! ҧ൓ग़඼ݕ஌ʢෳ਺υϝΠϯʣ ! Ձ֨αδΣετ ! ײಈग़඼ʢ৘ใͷࣗಈิ׬ʣ ! ഑ૹํ๏αδΣετ

    ! ը૾ݕࡧ ! etc.
  6. 6 ݱߦͷػցֶशαʔϏεͨͪ ! ҧ൓ग़඼ݕ஌ʢෳ਺υϝΠϯʣ = API + TF Serving !

    Ձ֨αδΣετ = API + TF Serving ! ײಈग़඼ʢ৘ใͷࣗಈิ׬ʣ = API + TF/ONNX Serving + Faiss ! ഑ૹํ๏αδΣετ = API + PostgreSQL ! ը૾ݕࡧ ɹ = API + TF Serving + Faiss ! etc. ଟ༷ͳΞʔΩςΫνϟͷਪ࿦αʔϏεΛKubernetesΫϥελʔ্Ͱӡ༻
  7. 7 ػցֶशαʔϏεͷಛੑ ! ୯७ʹܭࢉෛՙ͕ߴ͍ ! ෳ਺ίϯϙʔωϯτΛ૊Έ߹ΘͤͨෳࡶͳΞʔΩςΫνϟʹͳΓ΍͍͢ ◦ લॲཧɺޙॲཧΛߦ͏APIαʔόʔ ◦ Deep

    Learningͷਪ࿦αʔόʔʢTF Serving, ONNXRuntimeʣ ◦ ྨࣅ౓ݕࡧαʔόʔʢFaiss, NMSLIBʣ ◦ छʑͷσʔλϕʔεʢRedis, PostgreSQLʣ IstioͰαʔϏεϝογϡΛߏங͠ɺޮ཰తʹ؅ཧ
  8. 8 ͱ͸ʁ

  9. 9 ϚΠΫϩαʔϏεͱαʔϏεϝογϡ Control Plane Data Plane Service A Service Container

    Envoy Proxy Service B Service Container Envoy Proxy Service C Service Container Envoy Proxy Service D Service Container Envoy Proxy ! ϚΠΫϩαʔϏε܈ͷӡ༻ͱ࿈ܞΛޮ཰Խ͢Δ
  10. 10 Istioͷओͳػೳ ! ΞϓϦέʔγϣϯΞάϊεςΟοΫʹɺԼهͷػೳ͕ఏڙ͞ΕΔ ◦ τϥϑΟοΫ੍ޚ ◦ ௨৴಺༰΍ϝτϦΫεͷऔಘɺ؂ࢹ ◦ ೝূɺΞΫηε੍ݶɺ҉߸Խ

  11. 11 Istioͷओͳػೳ ! ΞϓϦέʔγϣϯΞάϊεςΟοΫʹɺԼهͷػೳ͕ఏڙ͞ΕΔ ◦ τϥϑΟοΫ੍ޚ ◦ ௨৴಺༰΍ϝτϦΫεͷऔಘɺ؂ࢹ ◦ ೝূɺΞΫηε੍ݶɺ҉߸Խ

  12. 12 ͕΋ͨΒ͢Ըܙ

  13. 13 ෛՙ΍ϨΠςϯγͷϞχλϦϯάɺࣗಈεέʔϦϯά ! Envoy͕औಘ͢ΔϝτϦΫεΛPrometheus + GrafanaͰ؂ࢹ ◦ ඵؒϦΫΤετ਺ ◦ ϨΠςϯγ

    ◦ CPUɾϝϞϦར༻཰ ! औಘͨ͠ϝτϦΫεΛݩʹɺHPAʢHorizontal Pod AutoscalerʣͰࣗಈε έʔϦϯά
  14. 14 ΧφϦΞσϓϩΠʹΑΔθϩμ΢ϯλΠϜϦϦʔε ! IstioͷVirtual ServiceʹΑΔτϥϑΟοΫ੍ޚΛ࢖༻ ◦ KubernetesͷServiceͷΈͰ͸ෆՄೳͳཻ౓Ͱ੍ޚՄೳ ! ࠶ֶशͨ͠Ϟσϧ΍ɺ৽نϑΟʔνϟʔΛ҆શʹϦϦʔεͰ͖Δ ◦

    ྲྀྔΛ0ʹ͢Δ͚ͩͰ҆શʹϩʔϧόοΫ - route: - destination: host: service-a.default.svc.cluster.local
 weight: 99 - destination: host: service-b.default.svc.cluster.local
 weight: 1 Service A Service B Virtual Service Service A Service B Virtual Service
  15. 15 ϛϥʔϦϯάʹΑΔຊ൪؀ڥͰͷαʔϏεݕূ ! IstioͷVirtual Service͸ϛϥʔϦϯάػೳ΋αϙʔτ ◦ ΦϦδφϧͷτϥϑΟοΫͷ׬શͳίϐʔΛྲྀೖͤ͞ΒΕΔ ! ࣮؀ڥͰͷϦΫΤετͰ҆શʹಈ࡞ݕূͰ͖Δ Service

    A Service B Virtual Service - route: - destination: host: service-a.default.svc.cluster.local
 weight: 100 mirror: host: service-b.default.svc.cluster.local
 mirror_percent: 100
  16. 16 ·ͱΊ

  17. 17 IstioͰػցֶशαʔϏεͷӡ༻Λ͍͍ײ͡ʹʂʂ ! ػցֶशαʔϏε͸ෳ਺ίϯϙʔωϯτ͔ΒͳΔෳࡶ͔ͭߴෛՙͳαʔϏ εʹͳΓ͕ͪ ! IstioʹΑΔαʔϏεϝογϡΛར༻͢Δ͜ͱͰɺޮ཰Α͘ෳ਺ͷػցֶश αʔϏεΛӡ༻ ◦ ΞϓϦέʔγϣϯଆʹखΛೖΕͳͯ͘Α͍

    ◦ ؂ࢹ΍ͦΕΛϕʔεʹͨࣗ͠ಈεέʔϦϯά ◦ τϥϑΟοΫ੍ޚΛར༻ͨ͠ݕূɺ҆શͳϦϦʔεɾϩʔϧόοΫ