$30 off During Our Annual Pro Sale. View Details »

Istioと共にマイクロサービスに立ち向かえ!

 Istioと共にマイクロサービスに立ち向かえ!

Aya (Igarashi) Ozawa

April 20, 2018
Tweet

More Decks by Aya (Igarashi) Ozawa

Other Decks in Technology

Transcript

  1. Japan Container Days v18.04
    *TUJPͱڞʹ.JDSPTFSWJDFT
    ʹཱͪ޲͔͑
    Aya Igarashi @Ladicle

    View Slide

  2. Software Engineer
    @Ladicle
    AYA IGARASHI

    View Slide

  3. Istio x KubernetesͰ͸
    ͦΕΒͷ໰୊ΛͲ͏ղܾ
    ͢Δͷ͔?
    ࿩͍ͨ͜͠ͱ
    Istioʹೖ໳͢Δʹ͸?
    ෳࡶʹͳͬͨ
    Microservicesύλʔϯ
    ͷγεςϜ͕࣋ͭ໰୊఺
    1 2 3
    ͭΒ͍….
    Αͦ͞͏
    ΍ͬͯΈΑ͏

    View Slide

  4. ฉ͍ͯ΄͍͜͠ͱ
    ͯ͠
    ͍Δ
    MicroservicesͰ։ൃ/ӡ༻Λ…
    IstioʹΑΔղܾํ๏ͱࠓ΍͍ͬͯΔ
    ํ๏Λൺֱ
    MicroservicesͰ͋Γ͕ͪͳ໰୊Λؾʹ
    ͱΊ͓ͯ͘
    ͜Ε
    ͔Β
    ͯ͠
    ͍Δ

    View Slide

  5. 2014
    Microservices
    https://martinfowler.com/articles/microservices.html
    James Lewis & Martin Fowler

    View Slide

  6. Monolithic vs. Microservices
    ༻్ɾ໨త͝ͱʹ
    খ͞ͳΞϓϦέʔγϣϯΛ࡞Δ
    Ұຕؠ(Monolitic)ͷΑ͏ʹ
    ҰͭͷΞϓϦέʔγϣϯΛ࡞Δ

    View Slide

  7. Scale-out ͠΍͍͢
    ૿΍͍ͨ͠ΞϓϦέʔγϣϯ͚ͩ
    Scale-outͰ͖Δ
    શͯΛҰؾʹ૿΍͢ or Scale-up
    ͷΈʹରԠ͍ͯ͠Δ

    View Slide

  8. Update
    มߋ͠΍͍͢
    Update
    Ұ෦ͷΞϓϦέʔγϣϯ͚ͩ
    ߋ৽͢Δ͜ͱ͕Ͱ͖Δ
    ModuleͷҰ෦ͷΈͷमਖ਼Ͱ΋
    ΞϓϦέʔγϣϯશମ͕ର৅

    View Slide

  9. େਓ਺Ͱ։ൃ͠΍͍͢
    API
    API
    νʔϜؒ͸APIΛఆٛ͠
    ಺෦ͷมߋ͸ಠཱ࣮ͯ͠ࢪͰ͖Δ
    ίϛϡχέʔγϣϯίετ͕ߴ͘
    ։ൃ଎౓͕Լ͕Δ

    View Slide

  10. NO
    SILVER
    BULLET

    View Slide

  11. ෳࡶʹͳΓଓ͚ΔγεςϜ
    2014
    2016
    201

    View Slide

  12. ෳࡶͳγεςϜ͕΋ͨΒ͢໰୊఺
    1
    2
    3
    4
    γεςϜͷશମ૾͕೺ѲͰ͖ͳ͍
    ΞϓϦέʔγϣϯؒͷ௨৴Λ੍ޚ͖͠Εͳ͍
    伴ͱূ໌ॻΛ؅ཧ͖͠Εͳ͍
    ো֐࣌ʹԿ͕ى͜Δ͔෼͔Βͳ͍
    Traffic Management
    Security
    Visualization
    Chaos Engineering

    View Slide

  13. Istio͕͜ΕΒͷ
    ໰୊Λղܾ͠·͢ʂ

    View Slide

  14. But,

    View Slide

  15. NO
    SILVER
    BULLET

    View Slide

  16. Istioͱ͸?
    “ෳࡶͳαʔϏεϝογϡͷ؅ཧ͕௥͍͔ͭͳ͍”
    ͱ͍͏໰୊Λղܾ͢ΔͨΊIBM, Google, Lyftʹ
    Αͬͯ։ൃ͞ΕͨOSSͰ͢ɻ
    https://github.com/istio/istio (latest: v0.7)
    ※ Istio͸ΪϦγϟޠͰൕͱ͍͏ҙຯ

    View Slide

  17. ΞϓϦέʔγϣϯؒͰ
    ϝογϡঢ়ʹ
    ௨৴͢ΔωοτϫʔΫͷ͜ͱ
    Service Meshͱ͸ʁ
    Mesh

    View Slide

  18. ͲͷΑ͏ʹServiceMeshΛ؅ཧ͍ͯ͠Δͷ͔?
    Pod
    App
    Envoy
    App
    Pod
    Pilot
    Mixer Auth
    Istio
    Controle Plane
    Data Plane
    Config data
    ֤ΞϓϦέʔγϣϯͷखલʹϓ
    ϩΩγαʔόΛஔ͖ɺ͜ͷϓϩ
    ΩγͷઃఆΛControle Plane͔
    ΒAPIܦ༝Ͱ੍ޚ͢Δ
    Envoy

    View Slide

  19. Envoyͱ͸?
    App
    Envoy
    C++11Ͱॻ͔Εͨ L4/L7 proxy
    LyftʹΑͬͯ։ൃ͞ΕͨOSS
    APIܦ༝ͷઃఆมߋ͕Մೳ
    ઃఆมߋ࣌ʹrestart͕ෆཁ
    HTTP/2 & GRPC ʹ΋ରԠ
    Proxy
    in/out-bound
    Traffic

    View Slide

  20. Problem 1
    ServiceMeshΛ੍ޚ͖͠Εͳ͍

    View Slide

  21. ໰୊1 ServiceMeshΛ੍ޚ͖͠Εͳ͍
    Complex
    ServiceMesh
    ΞϓϦέʔγϣϯͷίʔυʹServiceMesh
    ͷઃఆ͕ຒΊࠐ·Ε͍ͯΔ৔߹
    ↪︎ मਖ਼ຖʹ࠶σϓϩΠ͕ඞཁ
    ↪︎ ӡ༻/։ൃνʔϜͷ૒ํʹෛՙ

    View Slide

  22. ղܾࡦ1 ServiceMeshͷઃఆΛ෼཭
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    ….
    spec:
    # ϦΫΤετͷѼઌΛࢦఆ
    hosts:
    - myapp
    http:
    - match:
    - uri:
    # URI ʹ /v1/oldHome Λ͍࣋ͬͯΔ
    # ϦΫΤετʹϚον
    exact: /v1/oldHome
    redirect:
    # /v1/newHome ʹϦμΠϨΫτ
    uri: /v1/newHome
    authority: myapp.default.svc.cluster.local
    ServiceMeshͷઃఆ͕ΞϓϦέʔγϣ
    ϯίʔυʹґଘ͠ͳ͍
    L7LBʹΑͬͯॊೈͳϧʔςΟϯά͕
    Ͱ͖Δ
    ։ൃऀͱӡ༻ऀ͕෼ۀ͠΍͍͢
    ● ΧφϦΞʔϦϦʔεͰ҆৺ͳupgrade
    ● ϦΫΤετΛϓϩμΫγϣϯͱ
    εςʔδϯά؀ڥ΁ͷϛϥʔ
    ● pathʹΑͬͯΞϓϦέʔγϣϯΛ෼͚Δ
    ● ܞଳͱPCͳͲΫϥΠΞϯτʹԠͨ͡ৼΓ෼͚

    View Slide

  23. TrafficΛEnvoyʹྲྀ͢࢓૊Έ
    Init
    Istio
    App
    K8s
    1. Deploy App initContainers:
    - name: istio-init
    image: docker.io/istio/proxy_init
    ….
    containers:
    - name: istio-proxy
    image: docker.io/istio/proxy_debug
    securityContext:
    runAsUser: 1337
    args:
    - -p
    - “15001”
    ….
    - name: App
    image: docker.io/ladicle/myapp
    ......
    3.Change
    iptables rules
    Redirect
    In&Out bound Traffic
    UID: 1337
    2. Create istio-init
    4. Create App
    and istio-proxy
    proxy
    Istio

    View Slide

  24. istio-initʹΑΔiptablesͷมߋ
    istio-proxy@productpage-v1-5f9b797dfc-n6rn4:/$ sudo iptables -t nat -n -L -v
    Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
    pkts bytes target prot opt in out source destination
    0 0 ISTIO_REDIRECT all -- * * 0.0.0.0/0 0.0.0.0/0 /* istio/install-istio-prerouting */
    ….
    Chain OUTPUT (policy ACCEPT 1024 packets, 96005 bytes)
    pkts bytes target prot opt in out source destination
    21 1260 ISTIO_OUTPUT tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* istio/install-istio-output */
    ….
    Chain ISTIO_OUTPUT (1 references)
    pkts bytes target prot opt in out source destination
    0 0 ISTIO_REDIRECT all -- * lo 0.0.0.0/0 !127.0.0.1 /* istio/redirect-implicit-loopback */
    19 1140 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 owner UID match 1337 /* istio/bypass-envoy */
    0 0 RETURN all -- * * 0.0.0.0/0 127.0.0.1 /* istio/bypass-explicit-loopback */
    2 120 ISTIO_REDIRECT all -- * * 0.0.0.0/0 0.0.0.0/0 /* istio/redirect-default-outbound */
    Chain ISTIO_REDIRECT (3 references)
    pkts bytes target prot opt in out source destination
    2 120 REDIRECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* istio/redirect-to-envoy-port */ redir ports 15001
    Redirect all Traffic
    to Envoy
    Except own & lo

    View Slide

  25. K8s
    ServiceMeshͷઃఆΛม͑Δ
    App
    1. Call Rules API
    proxy
    Istio
    3. Call Envoy API
    Pilot
    2. Detect changes of
    service-account
    4. Update Envoy Config
    listeners:
    - address:
    socket_address:
    address: 0.0.0.0
    port_value: 15004
    filter_chains:
    - filters:
    ….
    http_filters:
    - config:
    default_destination_service: istio-pol
    service_configs:
    istio-policy.{{ .PodNamespace }}.svc
    disable_check_calls: true
    mixer_attributes:
    attributes:

    View Slide

  26. Problem 2
    伴ͱূ໌ॻΛ؅ཧ͖͠Εͳ͍

    View Slide

  27. ໰୊2 伴ͱূ໌ॻΛ؅ཧ͖͠Εͳ͍
    γεςϜʹΑͬͯ͸ΞϓϦέʔγϣϯؒ௨
    ৴ͷ҉߸Խ͸ඞਢ
    ΞϓϦέʔγϣϯ਺͕ଟ͘ͳΔͱɺ
    Difficult
    Key & Certificate
    Management
    ҰͭҰͭ伴΍ূ໌ॻͷੜ੒ɺ഑෍ɺߋ৽ɺ
    ແޮԽʹରԠ͢Δ͕࣌ؒͳ͘ͳΔ

    View Slide

  28. ղܾࡦ2 伴ͱূ໌ॻͷ؅ཧΛࣗಈԽ
    伴ͱূ໌ॻͷੜ੒, ഑෍,
    ߋ৽, ഇࢭʹରԠ͍ͯ͠Δ
    K8s
    1. Deploy App
    3. Generate self
    signed key & cert
    7. Push
    Secure Naming
    5. Deploy Pod
    with Secret
    2. Detect service
    account change
    CA
    Istio
    4. Store
    to Secret
    Pilot
    6. Watch
    Secure Naming
    Foo
    proxy
    Istio
    (SD)
    Mutual TLS(૬ޓTLS)Λαϙʔτ
    SAN: “spiffe://…/foo”
    mTLS
    Bar
    proxy
    Istio
    SAN: “spiffe://…/bar”
    ref: https://spiffe.io/

    View Slide

  29. Problem 3
    γεςϜͷશମ૾͕೺ѲͰ͖ͳ͍
    Observability, observability, observability!

    View Slide

  30. ໰୊3 γεςϜͷશମ૾͕೺ѲͰ͖ͳ͍
    ? ΞϓϦέʔγϣϯͷ਺͕େ͖͘ͳΓɺ೔ʑ
    Ͳ͔͜Ͱमਖ਼͕Ճ͑ΒΕ͍ͯΔ৔߹ɺ
    ↪︎ ߏ੒ਤͷखಈυΩϡϝϯτ͸஗Ε͕ͪ
    ↪︎ σόοά΍ো֐ղੳ͢Δʹ΋ϩά͚ͩͰ͸
    Ͳ͜·Ͱ੒ޭͨ͠ͷ͔೺Ѳͮ͠Β͍

    View Slide

  31. ໰୊3 ՄࢹԽπʔϧͱͷ࿈ܞ
    Metrics Traces
    Prometheus × Grafana Zipkin
    ServiceGraph
    Graphviz x Prometheus

    View Slide

  32. IstioͱPrometheus͕Ͳ͏࿈ܞ͢Δͷ͔?
    # Rule to send metric instances to a Prometheus handler
    apiVersion: "config.istio.io/v1alpha2"
    kind: rule
    metadata:
    name: mongoprom
    namespace: default
    spec:
    match: context.protocol == "tcp"
    && destination.service == "mongodb.default.svc.cluster.local"
    actions:
    - handler: mongohandler.prometheus
    instances:
    - mongoreceivedbytes.metric
    - mongosentbytes.metric
    # Configuration for a Prometheus han
    apiVersion: "config.istio.io/v1alpha2"
    kind: prometheus
    metadata:
    name: mongohandler
    namespace: default
    spec:
    metrics:
    - name: mongo_sent_bytes # Prometh
    instance_name: mongosentbytes.me
    kind: COUNTER
    label_names:
    - source_service
    - source_version
    - destination_version
    - name: mongo_received_bytes # Prom
    instance_name: mongoreceivedbytes
    ௨৴ܥͷMetrics͸উखʹ
    औͬͯ͘ΕΔ
    σϑΥϧτͷMetrics͔Β
    ServiceGraphΛੜ੒Ͱ͖Δ

    View Slide

  33. Distributed Tracingͱ͸?
    ϦΫΤετΛड͚औ͔ͬͯΒॲཧ͕׬ྃ͢Δ·ͰͷظؒΛSpanͱ͠
    ͯDAGͰදݱ͠ɺ͜ΕΛ࣌ܥྻʹฒ΂ͨ΋ͷΛτϨʔε݁Ռͱͯ͠
    දࣔ͢ΔɻશͯͷSpan͸໊લͱ։࢝࣌ؒɺܧଓ࣌ؒɺଞͷSpanͱ
    ͷRelationΛ͍࣋ͬͯΔɻ
    ΞϓϦέʔγϣϯؒͷϦΫΤετͷྲྀΕ΍࣌ؒΛՄࢹԽ͢Δπʔϧ
    Client
    LB
    Auth
    App
    DB
    1, 8
    2, 3
    4, 7
    5, 6
    Time

    View Slide

  34. IstioͱTraces͕Ͳ͏࿈ܞ͢Δͷ͔?
    ࣗಈతʹεύϯΛૹ৴Ͱ͖Δ
    τϨʔεશମΛؔ࿈͚ͮΔ͜ͱ͸Ͱ͖ͳ͍
    ҎԼͷϔομΛࢠʹ఻ൖͤ͞Δ͜ͱͰؔ࿈͚ͮͰ͖Δ
    • x-request-id
    • x-b3-traceid
    • x-b3-spanid
    • x-b3-parentspanid
    • x-b3-sampled
    • x-b3-flags
    • x-ot-span-context
    def getForwardHeaders(request):
    headers = {}
    user_cookie = request.cookies.get("user")
    if user_cookie:
    headers['Cookie'] = 'user=' + user_cookie
    incoming_headers = [ 'x-request-id',
    'x-b3-traceid',
    'x-b3-spanid',
    'x-b3-parentspanid',
    'x-b3-sampled',
    'x-b3-flags',
    'x-ot-span-context'
    ]
    for ihdr in incoming_headers:
    val = request.headers.get(ihdr)
    if val is not None:
    headers[ihdr] = val

    View Slide

  35. Mixer͕Backendͱ࿈ܞ͢Δ࢓૊Έ
    App
    proxy
    Istio
    K8s
    Update
    Mixer Config
    Mixer
    Adapters
    Report attributes
    Backends
    ΞϓϦέʔγϣϯίʔυͱBackendΛ஥հ
    MixterΛڬΉ͜ͱͰӡ༻ऀ੍͕ޚͰ͖Δ
    AdapterΛ௥Ճͯ͠ಠࣗͷBackendΛ૿΍ͤΔ

    View Slide

  36. MixtureͷϦΫΤετͷྲྀΕ

    View Slide

  37. Problem 4
    ো֐࣌ʹԿ͕ى͜Δ͔Θ͔Βͳ͍

    View Slide

  38. ໰୊4 ো֐࣌ʹԿ͕ى͜Δ͔Θ͔Βͳ͍
    γεςϜʹো֐͸͖ͭ΋ͷɻ
    ͔͠͠ɺো֐Λى͜͢ࢼݧ͸खಈͰ΍Δ
    ͱଓ͔ͣɺಠࣗʹ࣮૷͢Δͱଟ͘ͷίετ
    ͕͔͔Δɻ݁Ռɺޙճ͠ʹ͞Ε͕ͪ
    !?
    ↪︎ ো֐͕ى͜Δ·Ͱରࡦ͕ଧͨΕͳ͍

    View Slide

  39. ղܾࡦ4 Fault InjectionͰো֐ʹඋ͑Δ
    ର৅ൣғͱো֐಺༰Λࢦఆ͢Δ͚ͩͰ
    खܰʹো֐Λ࣮ݧͰ͖Δ apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
    name: reviews-route
    spec:
    ….
    fault:
    delay:
    percent: 10
    fixedDelay: 5s
    ….
    fault:
    abort:
    percent: 10
    httpStatus: 400
    ࢦఆ͞Εׂͨ߹ͷϦΫΤετʹରͯ͠
    Delay: ࢦఆ͞Ε͚ͨ࣌ؒͩ஗ΒͤΔ
    Abort: ֘౰ͷεςʔλείʔυΛฦ͢
    Chaos http2Error
    grpcStatus΋࣮૷༧ఆ

    View Slide

  40. ղܾࡦ4 Circuit BrakerͰো֐Λ࠷খݶʹ཈͑Δ
    apiVersion: config.istio.io/v1alpha2
    kind: RouteRule
    metadata:
    name: ratings-delay
    spec:

    httpFault:
    delay:
    percent: 10
    fixedDelay: 5s
    apiVersion: networking.istio.io/v1alpha3
    kind: DestinationRule

    trafficPolicy:
    connectionPool:
    tcp:
    maxConnections: 100
    http:
    http2MaxRequests: 1000
    maxRequestPerConnection: 10
    outlinerDetection:
    http:
    consecutiveErrors: 7
    Interval: 5m
    baseEjectionTime: 10
    ίʔυΛॻ͔ͣʹΤϥʔ͕ଓ͘ΞϓϦ
    έʔγϣϯ΁ͷϦΫΤετΛःஅ͢Δ
    Add
    x-envoy-overloaded
    Block Block
    Break

    View Slide

  41. ͓͞Β͍
    1
    2
    3
    4
    γεςϜͷશମ૾͕೺ѲͰ͖ͳ͍
    ServiceMeshΛ੍ޚ͖͠Εͳ͍
    伴ͱূ໌ॻΛ؅ཧ͖͠Εͳ͍
    ো֐࣌ʹԿ͕ى͜Δ͔Θ͔Βͳ͍
    ՄࢹԽπʔϧͱ࿈ܞ & ࣗಈԽΛαϙʔτ
    ServiceMeshͷઃఆΛ෼཭͢Δ
    ηΩϡΞ௨৴ͷͨΊͷ伴Λ؅ཧ͖͠Εͳ͍
    FIT & CircuitBrakerͰো֐ʹඋ͑Δ

    View Slide

  42. ଞʹ΋
    ৭ʑͰ͖·͢!
    Policy Enforcement
    Role Base Access Control
    Rate Limit
    Control Ingress Traffic
    Control Egress Traffic
    Integrate Bare Metal
    Logging
    Request Timeout
    Deploy to Eureka
    ɹɹɹɹɹMesos, CF

    View Slide

  43. Istioʹೖ໳͢Δ

    View Slide

  44. ೖ໳νϟʔτ
    C
    B
    D
    Start
    ͜ͷϓϨθϯ
    Λฉ͍ͯ֓ཁ
    Λ೺Ѳͨ͠
    Istio؀ڥΛ
    ηοτΞοϓ
    ͨ͠
    ڭ͑ͯཉ͍͠
    ͜ͷൃදͰ
    ؾʹͳΔ
    Topic͕͋Δ
    A
    YES
    YES
    YES
    Istio
    ॳ৺ऀͩ
    IstioΛ׬શʹ
    ཧղ͍ͯ͠Δ
    NO
    YES
    NO
    NO
    ৸ͯͨͱ͜Ζ͸
    SpeakerDeckΛcheck!
    https://speakerdeck.com/ladicle
    YES
    E
    QuickStartΛࢀߟʹ
    IstioΛΠϯετʔϧ
    https://istio.io/docs/setup/
    kubernetes/quick-start.html
    GKEͷQuick Start͕Φεεϝ
    ֘౰͢ΔTasksΛࢼ͢
    https://istio.io/docs/tasks/
    NO
    ໢ཏతͳGuidsͷ
    νϡʔτϦΞϧΛࢼ͢
    https://istio.io/docs/guides/

    View Slide

  45. ࠔͬͨ࣌͸ʁ
    1
    2
    3
    4
    ͓ࢼ͠தʹτϥϒͬͨ
    ࣭໰͕͋Δ
    2Ͱ΋ղܾ͠ͳ͍࣭໰͕͋Δ
    τϥγϡʔΨΠυΛࢀর https://istio.io/help/troubleshooting.html
    όάΛݟ͚ͭͨ
    ॆ࣮ͷFAQΛࢀর https://istio.io/help/faq
    ҆ఆͷStack Overflow https://stackoverflow.com/questions/tagged/istio
    GitHubͷIssue΁ https://github.com/istio/issues/issues/

    View Slide

  46. WE ARE
    HIRING!

    View Slide

  47. THANK YOU
    For you time & we’ll see you soon
    @ladicle

    View Slide