Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Istio Meets ML Ops - ML@Loft#9

Lain Matsuoka
December 19, 2019

Istio Meets ML Ops - ML@Loft#9

Lain Matsuoka

December 19, 2019
Tweet

More Decks by Lain Matsuoka

Other Decks in Programming

Transcript

  1. 1
    Meets MLOps
    ML@Loft #9
    Lain Matsuoka
    @lain_m21

    View Slide

  2. 2
    ! ॴଐɿAIνʔϜ ιϑτ΢ΣΞΤϯδχΞ
    ○ 2019/01ʙ
    ! ϝϧΧϦͰ΍ͬͨϓϩδΣΫτʢൈਮʣ
    ○ Ձ֨αδΣετ
    ○ τϨʔσΟϯάΧʔυը૾ೝࣝ
    ○ ݕࡧϦϥϯΩϯάʢόοΫΤϯυʣ
    ○ ΦϯϥΠϯ࣮ݧ؀ڥߏஙʢόοΫΤϯυʣ
    ! ϞσϦϯά͔ΒόοΫΤϯυɺΠϯϑϥ։ൃ·
    Ͱ෯޿͘୲౰
    Lain Matsuoka/দԬ ྰԻ

    View Slide

  3. 3
    ϝϧΧϦͷػցֶशαʔϏε
    Agenda
    Istioͱ͸ʁ
    Istio͕΋ͨΒ͢Ըܙ
    02
    03
    01

    View Slide

  4. 4
    ϝϧΧϦͷػցֶशαʔϏε

    View Slide

  5. 5
    ݱߦͷػցֶशαʔϏεͨͪ
    ! ҧ൓ग़඼ݕ஌ʢෳ਺υϝΠϯʣ
    ! Ձ֨αδΣετ
    ! ײಈग़඼ʢ৘ใͷࣗಈิ׬ʣ
    ! ഑ૹํ๏αδΣετ
    ! ը૾ݕࡧ
    ! etc.

    View Slide

  6. 6
    ݱߦͷػցֶशαʔϏεͨͪ
    ! ҧ൓ग़඼ݕ஌ʢෳ਺υϝΠϯʣ = API + TF Serving
    ! Ձ֨αδΣετ = API + TF Serving
    ! ײಈग़඼ʢ৘ใͷࣗಈิ׬ʣ = API + TF/ONNX Serving + Faiss
    ! ഑ૹํ๏αδΣετ = API + PostgreSQL
    ! ը૾ݕࡧ ɹ = API + TF Serving + Faiss
    ! etc.
    ଟ༷ͳΞʔΩςΫνϟͷਪ࿦αʔϏεΛKubernetesΫϥελʔ্Ͱӡ༻

    View Slide

  7. 7
    ػցֶशαʔϏεͷಛੑ
    ! ୯७ʹܭࢉෛՙ͕ߴ͍
    ! ෳ਺ίϯϙʔωϯτΛ૊Έ߹ΘͤͨෳࡶͳΞʔΩςΫνϟʹͳΓ΍͍͢
    ○ લॲཧɺޙॲཧΛߦ͏APIαʔόʔ
    ○ Deep Learningͷਪ࿦αʔόʔʢTF Serving, ONNXRuntimeʣ
    ○ ྨࣅ౓ݕࡧαʔόʔʢFaiss, NMSLIBʣ
    ○ छʑͷσʔλϕʔεʢRedis, PostgreSQLʣ
    IstioͰαʔϏεϝογϡΛߏங͠ɺޮ཰తʹ؅ཧ

    View Slide

  8. 8
    ͱ͸ʁ

    View Slide

  9. 9
    ϚΠΫϩαʔϏεͱαʔϏεϝογϡ
    Control Plane
    Data Plane
    Service A
    Service
    Container
    Envoy
    Proxy
    Service B
    Service
    Container
    Envoy
    Proxy
    Service C
    Service
    Container
    Envoy
    Proxy
    Service D
    Service
    Container
    Envoy
    Proxy
    ! ϚΠΫϩαʔϏε܈ͷӡ༻ͱ࿈ܞΛޮ཰Խ͢Δ

    View Slide

  10. 10
    Istioͷओͳػೳ
    ! ΞϓϦέʔγϣϯΞάϊεςΟοΫʹɺԼهͷػೳ͕ఏڙ͞ΕΔ
    ○ τϥϑΟοΫ੍ޚ
    ○ ௨৴಺༰΍ϝτϦΫεͷऔಘɺ؂ࢹ
    ○ ೝূɺΞΫηε੍ݶɺ҉߸Խ

    View Slide

  11. 11
    Istioͷओͳػೳ
    ! ΞϓϦέʔγϣϯΞάϊεςΟοΫʹɺԼهͷػೳ͕ఏڙ͞ΕΔ
    ○ τϥϑΟοΫ੍ޚ
    ○ ௨৴಺༰΍ϝτϦΫεͷऔಘɺ؂ࢹ
    ○ ೝূɺΞΫηε੍ݶɺ҉߸Խ

    View Slide

  12. 12
    ͕΋ͨΒ͢Ըܙ

    View Slide

  13. 13
    ෛՙ΍ϨΠςϯγͷϞχλϦϯάɺࣗಈεέʔϦϯά
    ! Envoy͕औಘ͢ΔϝτϦΫεΛPrometheus + GrafanaͰ؂ࢹ
    ○ ඵؒϦΫΤετ਺
    ○ ϨΠςϯγ
    ○ CPUɾϝϞϦར༻཰
    ! औಘͨ͠ϝτϦΫεΛݩʹɺHPAʢHorizontal Pod AutoscalerʣͰࣗಈε
    έʔϦϯά

    View Slide

  14. 14
    ΧφϦΞσϓϩΠʹΑΔθϩμ΢ϯλΠϜϦϦʔε
    ! IstioͷVirtual ServiceʹΑΔτϥϑΟοΫ੍ޚΛ࢖༻
    ○ KubernetesͷServiceͷΈͰ͸ෆՄೳͳཻ౓Ͱ੍ޚՄೳ
    ! ࠶ֶशͨ͠Ϟσϧ΍ɺ৽نϑΟʔνϟʔΛ҆શʹϦϦʔεͰ͖Δ
    ○ ྲྀྔΛ0ʹ͢Δ͚ͩͰ҆શʹϩʔϧόοΫ
    - route:
    - destination:
    host: service-a.default.svc.cluster.local

    weight: 99
    - destination:
    host: service-b.default.svc.cluster.local

    weight: 1
    Service A
    Service B
    Virtual
    Service
    Service A
    Service B
    Virtual
    Service

    View Slide

  15. 15
    ϛϥʔϦϯάʹΑΔຊ൪؀ڥͰͷαʔϏεݕূ
    ! IstioͷVirtual Service͸ϛϥʔϦϯάػೳ΋αϙʔτ
    ○ ΦϦδφϧͷτϥϑΟοΫͷ׬શͳίϐʔΛྲྀೖͤ͞ΒΕΔ
    ! ࣮؀ڥͰͷϦΫΤετͰ҆શʹಈ࡞ݕূͰ͖Δ
    Service A
    Service B
    Virtual
    Service
    - route:
    - destination:
    host: service-a.default.svc.cluster.local

    weight: 100
    mirror:
    host: service-b.default.svc.cluster.local

    mirror_percent: 100

    View Slide

  16. 16
    ·ͱΊ

    View Slide

  17. 17
    IstioͰػցֶशαʔϏεͷӡ༻Λ͍͍ײ͡ʹʂʂ
    ! ػցֶशαʔϏε͸ෳ਺ίϯϙʔωϯτ͔ΒͳΔෳࡶ͔ͭߴෛՙͳαʔϏ
    εʹͳΓ͕ͪ
    ! IstioʹΑΔαʔϏεϝογϡΛར༻͢Δ͜ͱͰɺޮ཰Α͘ෳ਺ͷػցֶश
    αʔϏεΛӡ༻
    ○ ΞϓϦέʔγϣϯଆʹखΛೖΕͳͯ͘Α͍
    ○ ؂ࢹ΍ͦΕΛϕʔεʹͨࣗ͠ಈεέʔϦϯά
    ○ τϥϑΟοΫ੍ޚΛར༻ͨ͠ݕূɺ҆શͳϦϦʔεɾϩʔϧόοΫ

    View Slide