Istio Meets ML Ops - ML@Loft#9

Transcript

1. 1
   Meets MLOps
   ML@Loft #9
   Lain Matsuoka
   @lain_m21
   

   View Slide

2. 2
   ! ॴଐɿAIνʔϜ ιϑτ΢ΣΞΤϯδχΞ
   ○ 2019/01ʙ
   ! ϝϧΧϦͰ΍ͬͨϓϩδΣΫτʢൈਮʣ
   ○ Ձ֨αδΣετ
   ○ τϨʔσΟϯάΧʔυը૾ೝࣝ
   ○ ݕࡧϦϥϯΩϯάʢόοΫΤϯυʣ
   ○ ΦϯϥΠϯ࣮ݧ؀ڥߏஙʢόοΫΤϯυʣ
   ! ϞσϦϯά͔ΒόοΫΤϯυɺΠϯϑϥ։ൃ·
   Ͱ෯޿͘୲౰
   Lain Matsuoka/দԬ ྰԻ
   

   View Slide

3. 3
   ϝϧΧϦͷػցֶशαʔϏε
   Agenda
   Istioͱ͸ʁ
   Istio͕΋ͨΒ͢Ըܙ
   02
   03
   01
   

   View Slide

4. 4
   ϝϧΧϦͷػցֶशαʔϏε
   

   View Slide

5. 5
   ݱߦͷػցֶशαʔϏεͨͪ
   ! ҧ൓ग़඼ݕ஌ʢෳ਺υϝΠϯʣ
   ! Ձ֨αδΣετ
   ! ײಈग़඼ʢ৘ใͷࣗಈิ׬ʣ
   ! ഑ૹํ๏αδΣετ
   ! ը૾ݕࡧ
   ! etc.
   

   View Slide

6. 6
   ݱߦͷػցֶशαʔϏεͨͪ
   ! ҧ൓ग़඼ݕ஌ʢෳ਺υϝΠϯʣ = API + TF Serving
   ! Ձ֨αδΣετ = API + TF Serving
   ! ײಈग़඼ʢ৘ใͷࣗಈิ׬ʣ = API + TF/ONNX Serving + Faiss
   ! ഑ૹํ๏αδΣετ = API + PostgreSQL
   ! ը૾ݕࡧ ɹ = API + TF Serving + Faiss
   ! etc.
   ଟ༷ͳΞʔΩςΫνϟͷਪ࿦αʔϏεΛKubernetesΫϥελʔ্Ͱӡ༻
   

   View Slide

7. 7
   ػցֶशαʔϏεͷಛੑ
   ! ୯७ʹܭࢉෛՙ͕ߴ͍
   ! ෳ਺ίϯϙʔωϯτΛ૊Έ߹ΘͤͨෳࡶͳΞʔΩςΫνϟʹͳΓ΍͍͢
   ○ લॲཧɺޙॲཧΛߦ͏APIαʔόʔ
   ○ Deep Learningͷਪ࿦αʔόʔʢTF Serving, ONNXRuntimeʣ
   ○ ྨࣅ౓ݕࡧαʔόʔʢFaiss, NMSLIBʣ
   ○ छʑͷσʔλϕʔεʢRedis, PostgreSQLʣ
   IstioͰαʔϏεϝογϡΛߏங͠ɺޮ཰తʹ؅ཧ
   

   View Slide

8. 8
   ͱ͸ʁ
   

   View Slide

9. 9
   ϚΠΫϩαʔϏεͱαʔϏεϝογϡ
   Control Plane
   Data Plane
   Service A
   Service
   Container
   Envoy
   Proxy
   Service B
   Service
   Container
   Envoy
   Proxy
   Service C
   Service
   Container
   Envoy
   Proxy
   Service D
   Service
   Container
   Envoy
   Proxy
   ! ϚΠΫϩαʔϏε܈ͷӡ༻ͱ࿈ܞΛޮ཰Խ͢Δ
   

   View Slide

10. 10
    Istioͷओͳػೳ
    ! ΞϓϦέʔγϣϯΞάϊεςΟοΫʹɺԼهͷػೳ͕ఏڙ͞ΕΔ
    ○ τϥϑΟοΫ੍ޚ
    ○ ௨৴಺༰΍ϝτϦΫεͷऔಘɺ؂ࢹ
    ○ ೝূɺΞΫηε੍ݶɺ҉߸Խ
    

    View Slide

11. 11
    Istioͷओͳػೳ
    ! ΞϓϦέʔγϣϯΞάϊεςΟοΫʹɺԼهͷػೳ͕ఏڙ͞ΕΔ
    ○ τϥϑΟοΫ੍ޚ
    ○ ௨৴಺༰΍ϝτϦΫεͷऔಘɺ؂ࢹ
    ○ ೝূɺΞΫηε੍ݶɺ҉߸Խ
    

    View Slide

12. 12
    ͕΋ͨΒ͢Ըܙ
    

    View Slide

13. 13
    ෛՙ΍ϨΠςϯγͷϞχλϦϯάɺࣗಈεέʔϦϯά
    ! Envoy͕औಘ͢ΔϝτϦΫεΛPrometheus + GrafanaͰ؂ࢹ
    ○ ඵؒϦΫΤετ਺
    ○ ϨΠςϯγ
    ○ CPUɾϝϞϦར༻཰
    ! औಘͨ͠ϝτϦΫεΛݩʹɺHPAʢHorizontal Pod AutoscalerʣͰࣗಈε
    έʔϦϯά
    

    View Slide

14. 14
    ΧφϦΞσϓϩΠʹΑΔθϩμ΢ϯλΠϜϦϦʔε
    ! IstioͷVirtual ServiceʹΑΔτϥϑΟοΫ੍ޚΛ࢖༻
    ○ KubernetesͷServiceͷΈͰ͸ෆՄೳͳཻ౓Ͱ੍ޚՄೳ
    ! ࠶ֶशͨ͠Ϟσϧ΍ɺ৽نϑΟʔνϟʔΛ҆શʹϦϦʔεͰ͖Δ
    ○ ྲྀྔΛ0ʹ͢Δ͚ͩͰ҆શʹϩʔϧόοΫ
    - route:
    - destination:
    host: service-a.default.svc.cluster.local

    weight: 99
    - destination:
    host: service-b.default.svc.cluster.local

    weight: 1
    Service A
    Service B
    Virtual
    Service
    Service A
    Service B
    Virtual
    Service
    

    View Slide

15. 15
    ϛϥʔϦϯάʹΑΔຊ൪؀ڥͰͷαʔϏεݕূ
    ! IstioͷVirtual Service͸ϛϥʔϦϯάػೳ΋αϙʔτ
    ○ ΦϦδφϧͷτϥϑΟοΫͷ׬શͳίϐʔΛྲྀೖͤ͞ΒΕΔ
    ! ࣮؀ڥͰͷϦΫΤετͰ҆શʹಈ࡞ݕূͰ͖Δ
    Service A
    Service B
    Virtual
    Service
    - route:
    - destination:
    host: service-a.default.svc.cluster.local

    weight: 100
    mirror:
    host: service-b.default.svc.cluster.local

    mirror_percent: 100
    

    View Slide

16. 16
    ·ͱΊ
    

    View Slide

17. 17
    IstioͰػցֶशαʔϏεͷӡ༻Λ͍͍ײ͡ʹʂʂ
    ! ػցֶशαʔϏε͸ෳ਺ίϯϙʔωϯτ͔ΒͳΔෳࡶ͔ͭߴෛՙͳαʔϏ
    εʹͳΓ͕ͪ
    ! IstioʹΑΔαʔϏεϝογϡΛར༻͢Δ͜ͱͰɺޮ཰Α͘ෳ਺ͷػցֶश
    αʔϏεΛӡ༻
    ○ ΞϓϦέʔγϣϯଆʹखΛೖΕͳͯ͘Α͍
    ○ ؂ࢹ΍ͦΕΛϕʔεʹͨࣗ͠ಈεέʔϦϯά
    ○ τϥϑΟοΫ੍ޚΛར༻ͨ͠ݕূɺ҆શͳϦϦʔεɾϩʔϧόοΫ
    

    View Slide