Managing Microservices with Istio Service Mesh

Developing and managing hundreds (or maybe thousands) of microservices at scale is a challenge for both development and operations teams. We have seen over the last few years the appearance of new frameworks dedicated to delivering Cloud Native applications by providing a set of (out of the box) building blocks. Most of these frameworks integrate microservices concerns at the code level. Recently, we have seen the emergence of a new pattern known as sidecar or proxy promoting to push all these common concerns outside of the business code and provides them on the edge by integrating a new layer to the underlying platform called Service Mesh. Istio is one of the leading Service Mesh implementing sidecar pattern. We will go during the presentation throw the core concepts behind Istio, the capabilities that provide to manage, secure and observe microservices and how it gives a new breath for both developers and operations.

Transcript

1. Managing microservices with Istio Service Mesh Rafik Harabi
2. About me • Prev. Solution Architect at INNOVSQUARE - Digital Transformation and Cloud Migration. • Currently, Technical Account Manager with Sysdig - Cloud Native Observability and Security • #Kubernetes #CloudNative are my daily routines. @rafik8_ https://fr.linkedin.com/in/rafikharabi
3. Microservices challenges Network Reliability Fault tolerance and resiliency Monitoring and Observability Security
4. Microservices challenges, deep dive Network Reliability Service have to handle the network facts: • Network latency / bandwidth • Transport cost • Topology and administration Fault Tolerance Service have to be able to handle outright failure and timeouts: • Avoid cascading failure • Circuit breaking • Retries Monitoring We have to: • monitor the delivered microservices and their interactions • Trace requests and identify potential hotspots Security We have to: • Secure communication between microservices • Service to service authentication and authorization
5. Challenges: Real life case Source: https://twitter.com/JackKleeman/status/1190354757308862468
6. The evolution of microservices frameworks: from NetFlix OSS to Istio
7. 2011 NetFlix OSS first microservices patterns and libraries open-sourced 2013 Spring Cloud Enterprise microservice framework for Java 2014 Docker Containerization 2015 Kubernetes Workload orchestration 2018 Istio Service mesh
8. Microservices challenges Challenge 1 Challenge 2 Challenge 3 - N to N communications. - Distributed software interconnection and troubleshooting is hard. - Containers should stay thin and platform agnostic. - Upgrade of polyglot microservices is hard at scale.
9. Microservices building blocks Challenge 1 Challenge 3 Configuration Service Service Registry / Discovery Circuit Breaker / Retry Rate Limiting API Gateway Load Balancing / Intelligent Routing Authentication & Authorization Monitoring Distributed tracing Event Driven Messaging (Async) Log Aggregation Audit
10. Microservices building blocks Challenge 3 Business Value Configuration Service Service Registry / Discovery Circuit Breaker / Retry Rate Limiting Event Driven Messaging (Async) Audit Load Balancing / Intelligent Routing API Gateway Authentication & Authorization Monitoring Distributed tracing Log Aggregation } Business outcome
11. Code oriented frameworks Challenge 3 Service A Service B Business logic Business logic Circuit Breaker Rate limiting Tracing Metrics Circuit Breaker Rate limiting Tracing Metrics
12. Code oriented pattern Challenge 1 Challenge 3 Configuration Service Service Registry / Discovery Circuit Breaker/Retry Rate Limiting API Gateway Load Balancing / Intelligent Routing Authentication & Authorization Monitoring Distributed tracing Event Driven Messaging (Async) Log Aggregation Audit Business Service Foundation Monitoring and Observability Communication Business Values
13. Code oriented solutions limits - Language oriented. - Error prone (implementation). - Hard to upgrade each microservice when system grow. - Add technical challenges and duties to development teams. - Different teams in the same organization may have different implementations. - Each team should maintien his implementation. Microservices challenges need to be solved uniformly
14. Desired state - Keep microservice concerns separate from the business logic. - The network should be transparent to applications. - Developers should focus on delivering business capabilities and not implementing microservices common concerns. - Microservices interconnection should be language agnostic. - Easy to upgrade solution.
15. Service Mesh Definition A service mesh is a dedicated infrastructure layer for handling service-to-service communication. It’s responsible for the reliable delivery of requests through the complex topology of services that comprise a modern, cloud native application. buoyant.io
16. Service Mesh The design Each service will have its own proxy service and all these proxy services together form the “Service Mesh”. All the requests to and from each service will go through the mesh proxies. Proxies are also known as sidecars.
17. Sidecar pattern Service A Business logic Circuit Breaker Rate limiting Tracing Metrics Proxy Service B Business logic Circuit Breaker Rate limiting Tracing Metrics Proxy Injected Network concerns become transparent Service to service communication
18. Service Mesh landscape Source: https://landscape.cncf.io/category=service-mesh
19. Why Istio? Source: https://twitter.com/TorstenVolk/status/1197243464494108673/ Twitter trends at KubeCon/CloudNative North America 2019 Istio
20. History of Istio - Envoy proxy (Istio data plane) created by Lyft and open-sourced in 2016. - Lyft, IBM and Google launch the project in May 2017. - First major version released in July 2018. - Current version 1.7
21. Istio goal Develop an open technology that provides a uniform way to connect, secure, manage and monitor a network of microservices regardless of the platform source or vendor.
22. Solution Istio Promises • Focus on business logic and spent less time with common concerns. • No change in the service code. • Central configuration management. • Easy to upgrade • Security
23. Istio does: - Service discovery - Load Balancing & Intelligent Routing - Resiliency: Circuit Breaker & Retry - Rate Limiting - Authentication and Authorization - Service to Service mTLS - Policy enforcement - Observability - Monitoring metrics - Distributed tracing - Event Driven Asynchronous communication - Service Orchestration Istio does not:
24. Sidecar pattern Challenge 1 Challenge 3 Configuration Service Service Registry / Discovery Circuit Breaker/Retry Rate Limiting API Gateway Authentication & Authorization Monitoring Distributed tracing Event Driven Messaging (Async) Log Aggregation Audit Business Service Foundation Monitoring and Observability Communication Business Values Load Balancing / Intelligent Routing Business Service Business Service
25. Challenge 1 Challenge 2 Challenge 3
26. Istio building blocks- Control Plane 1/2 Component Description Pilot Responsible for service discovery and for configuring the Envoy sidecar proxies Citadel Automated key and certificate management Mixer Istio-Policy: policy enforcement Istio-Telemetry: gather telemetry data Galley Configuration ingestion for istio components Ingress Gateway manage inbound connection to the service mesh Egress Gateway manage outbound connection from the service mesh
27. Istio building blocks - Addons 1/2 Component Description Prometheus Metrics collection Grafana Monitoring dashboard Jaeger Distributed tracing Kiali Observability dashboard & Service Service Graph Service dependencies
29. Install Istio • Installation Options: ◦ Helm 3 chart ◦ Operator Install Using Helm: Prepare: export ISTIO_VERSION=1.7 curl -L https://git.io/getLatestIstio | sh - Check: istioctl verify-install Install: istioctl install --set profile=demo
30. Enabling Service Mesh 1/4 • Istio provide 3 options: ◦ Automatic sidecar injection: label namespace istioinjection=enabled ◦ Manual sidecar injection: istioctl kube-inject -f deployment.yaml ◦ Single pod sidecar injection: istioctl experimental add-to-mesh service
31. Enabling Service Mesh 2/4 istio-init istio-proxy Service container 1 2 3 kubectl describe po/pod-id -n pod-name
32. Enabling Service Mesh 3/4 Sidecar initialization (istio-init)
33. kubectl logs po/pod-id -c istio-init -n pod-name Sidecar initialization (istio-init) Enabling Service Mesh 3/4
34. Service Mesh Patterns Traffic Management Observability and Monitoring Security
35. Traffic Management
36. Service Discovery Challenge 1 Challenge 2 Challenge 3 Kubernetes provides service discovery, why do I need an extra layer ? Istio supports: - HTTP L7 filter - HTTP L7 routing (based on http headers and cookies) - First class HTTP/2 - gRPC support - Fine-grained traffic splitting
37. Traffic Management / Splitting Challenge 1 Challenge 3 EDS: Endpoint Discovery Service CDS: Cluster Discovery Service RDS: Route Discovery Service LDS: Listener Discovery Service istioctl proxy-status istioctl proxy-config clusters -n namespace pod-id Product Page Proxy Details Proxy Pilot xDS Sync xDS Sync
38. Traffic Splitting • Achieve affinity by using Cookie, Header, or Source IP. • Each service can have any number of versions (a.k.a. subsets). • Pilot translates high-level rules into low-level configurations and distributes this config to Envoy instances. • Pilot uses three types of configuration resources to manage traffic within its service mesh: Virtual Services, Destination Rules, and Service Entries.
39. Traffic Splitting
40. Traffic Splitting apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: service-a spec: host: service-a subsets: - name: v1 labels: version: version-1 - name: v2 labels: version: version-2 apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: service-a spec: hosts: - service-a http: - route: - destination: host: service-a subset: v1 weight: 80 - destination: host: service-a subset: v2 weight: 20
41. A/B testing - Canary release 100% 0%
42. A/B testing - Canary release 100% 0% 80% 20%
43. A/B testing - Canary release 100% 0% 80% 20% 50% 50%
44. A/B testing - Canary release 100% 0% 80% 20% 50% 50% 100% 0%
45. Traffic mirroring
46. Ingress Gateway Challenge 1 Challenge 2 Challenge 3 Ingress Define access to services from the outside the service mesh:
47. Ingress Gateway Challenge 1 Challenge 2 Challenge 3 Gateway: defines a load balancer operating at the edge of the mesh receiving incoming (Ingress) or outgoing (Egress) HTTP/TCP connections. VirtualService: defines a set of traffic routing rules to apply when a host (service) is addressed. DestinationRule: defines policies that apply to traffic intended for a service after routing has occurred.
48. Ingress Gateway Challenge 1 Challenge 2 Challenge 3 apiVersion: networking.istio.io/v1beta1 kind: Gateway metadata: name: service-gateway spec: selector: istio: ingressgateway servers: - port: number: 80 name: http protocol: HTTP hosts: - "service-name.my-domain.com" apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: service-public spec: hosts: - "*" gateways: - service-gateway http: - match: - uri: exact: /service-url route: - destination: host: service-a port: number: 9080 Service URL Target Kubernetes service Target service Port
49. Egress Gateway Challenge 1 Challenge 2 Challenge 3 Egress gateway controls outbound connection and managed authorization in a declarative way.
50. Challenge 1 Challenge 2 Challenge 3 Egress Gateway Challenge 1 Challenge 2 Challenge 3 apiVersion: networking.istio.io/v1beta1 kind: ServiceEntry metadata: name: service-a-serviceentry spec: hosts: - my-external-service-dns servers: - port: number: 3306 name: tcp-mysql protocol: TCP location:MESH_EXTERNAL apiVersion: networking.istio.io/v1beta1 kind: Gateway metadata: name: istio-egressgateway spec: Selector: ` istio: egressgateway servers: - port: number: 3306 name: tcp-mysql protocol: TCP location:my-external-service-dns
51. Fault Injection Inject faults to test the resiliency of your application: Delays: Delays are timing failures such us network latency or overloaded upstreams. Aborts: Aborts are crash failures such as HTTP error codes or TCP connection failures.
52. Fault Injection apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: service-a-503 spec: hosts: - service-a http: - route: - destination: host: service-a fault: abort: percent: 20 httpStatus: 503 apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: service-a-delay-5 spec: hosts: - service-a http: - route: - destination: host: service-a fault: delay: percent: 100 fixedDelay: 5s Delay Abort
53. Circuit Breaker Circuit breaking allows to write applications that limit the impact of failures, latency spikes, and other network effects.
54. Circuit Breaker apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: service-a spec: host: service-a trafficPolicy: connectionPool: tcp: maxConnections: 1 http: http1MaxPendingRequests: 10 maxRequestsPerConnection: 2 outlierDetection: consecutiveErrors: 3 interval: 1s baseEjectionTime: 3m maxEjectionPercent: 100
55. Retry strategy - Enable automatic retry when a temporary network issue occurs. - Timeout could be also specified.
56. Retry strategy Enable automatic retry when a temporary network issue occurs. Timeout could be also specified. apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: service-a spec: hosts: - service-a http: - route: - destination: host: service-a retries: attempts: 3 perTryTimeout: 2s
57. Security
58. Mutual TLS traffic is encrypted on transit between services: - Service Level - Namespace Level - Mesh Level apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: default namespace: my-namespace spec: mtls: mode:STRICT
59. Service to Service Authorization
60. Service to Service Authorization apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: allow-read namespace: default spec: Selector: matchLabels: app: service-b action: Allow rules: - from: - source: principals: ["cluster.local/ns/default/sa/service-a"] to: - operation: methods: ["POST"] when: - key: request.headers[user-role] values: ["operator", "internal"]
61. Observability and Monitoring
62. Dashboard Challenge 1 Challenge 2 Challenge 3 • Monitoring: Prometheus • Observability: Kiali • Tracing: Jaeger/Zipkin • Service graph: Kiali istioctl dashbord Usage: istioctl dashboard [flags] istioctl dashboard [command] Aliases: dashboard, dash, d Available Commands: controlz Open ControlZ web UI envoy Open Envoy admin web UI grafana Open Grafana web UI jaeger Open Jaeger web UI kiali Open Kiali web UI prometheus Open Prometheus web UI zipkin Open Zipkin web UI
63. Dashboard - Grafana
64. Dashboard - Grafana
65. Dashboard - Kiali
66. Dashboard - Kiali
67. Dashboard - Jaeger
68. Third Party Integration
69. Telemetry third party integration
70. Policy third party integration
71. Debugging - using istioctl kubectl logs po/POD-ID -c istio-init -n namespace istioctl proxy-status istioctl proxy-config clusters -n namespace POD-ID istioctl proxy-config listeners -n namespace POD-ID
72. Advanced topics (not covered) - Multi-cluster support - Mesh Expansion
73. Service Mesh Maturity Model
74. Deep dive with istio - Workshops - IBM workshop: https://istio101.gitbook.io/lab/ - Redhat interactive learn: https://learn.openshift.com/servicemesh - Redhat developer demos: https://github.com/redhat-developer-demos/istio-tutorial/ - Istio by example, Ray Tsang: https://github.com/saturnism/istio-by-example-java - Service Mesh Workshop, Rafik Harabi @SRECon Europe 2019: https://www.istioworkshop.io
75. Deep dive with Istio - Ebooks Istio Explained Istio in action Istio Up and Running
76. “Istio can (and should) be adopted incrementally.” Christian Posta, Istio in Action
77. Thank you! Any questions ?