Managing microservices with Istio Service Mesh

Developing and managing hundreds (or maybe thousands) of microservices at scale is a challenge for both development and operations teams.

We have seen over the last years the appearance of new frameworks dedicated to deliver ‘Cloud Native’ applications by providing a set of (out of box) building blocks. Most of these frameworks integrate microservices concerns at the code level.
Recently, we have seen the emerging of a new pattern known as sidecar or proxy promoting to push all these common concerns outside of the business code and provides them on the edge by integrate a new layer to the underlying platform called Service Mesh.

Istio is one of the leading Service Mesh implementing sidecar pattern.
We will go during the presentation throw the core concepts behind Istio, the capabilities that provides to manage, secure and observe microservices and how it gives a new breath for both developers and operations.
The presentation will be guided by a sequence of demo exposing Istio capabilities.

Transcript

1. Managing microservices with Istio Service Mesh Rafik Harabi, INNOVSQUARE CNCF Tunisia Meetup 12/2018
2. Who I am? • Solution Architect at INNOVSQUARE focusing on Digital Transformation and Go2Cloud. • Deloitte UK (London) Associate • #Kubernetes, #Istio and #SpringPlatform are my daily routines. @rafik8_ https://fr.linkedin.com/in/rafikharabi
3. Moving to microservices network challenges Network Reliability Fault tolerance and resiliency Monitoring and Observability
4. Challenges deep-dive Network Reliability Service have to handle the network facts: • Network latency / bandwidth • Transport cost • Topology and administration Fault Tolerance Service have to be able to handle outright failure and timeouts: • Avoid cascading failure • Retries • Circuit breaking Monitoring We have to: • monitor the delivered microservices and their interactions • Trace requests and identify potential hotspots
5. The evolution of microservices frameworks: from NetFlix OSS to Istio
6. 2011 NetFlix OSS first microservices patterns and libraries open-sourced 2013 Spring Cloud Enterprise microservice framework for Java 2014 Docker Containerization 2015 Kubernetes Workload orchestration 2018 Istio Service mesh
7. Microservices challenges Challenge 1 Challenge 2 Challenge 3 - N to N communications. - Distributed software interconnection and troubleshooting is hard. - Containers should stay thin and platform agnostic. - Upgrade of polyglot microservices is hard at scale.
8. Microservices building blocks Challenge 1 Challenge 3 Configuration Service Service Registry / Discovery Circuit Breaker / Retry Rate Limiting API Gateway Load Balancing / Intelligent Routing Authentication & Authorization Monitoring Distributed tracing Event Driven Messaging (Async) Log Aggregation Audit
9. Microservices building blocks Challenge 3 Business Value Configuration Service Service Registry / Discovery Circuit Breaker / Retry Rate Limiting Event Driven Messaging (Async) Audit Load Balancing / Intelligent Routing API Gateway Authentication & Authorization Monitoring Distributed tracing Log Aggregation
10. Code oriented frameworks Challenge 3 Service A Service B Business logic Business logic Circuit Breaker Rate limiting Tracing Metrics Circuit Breaker Rate limiting Tracing Metrics
11. Code oriented pattern Challenge 1 Challenge 3 Configuration Service Service Registry / Discovery Circuit Breaker/Retry Rate Limiting API Gateway Load Balancing / Intelligent Routing Authentication & Authorization Monitoring Distributed tracing Event Driven Messaging (Async) Log Aggregation Audit Business Service Foundation Monitoring and Observability Communication Business Values
12. Code oriented solutions limits - Language oriented. - Error prone (implementation). - Hard to upgrade each microservice when system grow. - Add technical challenges and duties to development teams. - Different team in the same organization may have different implementation. - Each team should maintien his implementation.
13. Desired state - Keep microservice concerns separate from the business logic. - The network should be transparent to applications. - Developers should focus on delivering business capabilities and not implementing microservices common concerns. - Microservices interconnection should be language agnostic. - Easy to upgrade solution.
14. Service Mesh Definition A service mesh is a dedicated infrastructure layer for handling service-to-service communication. It’s responsible for the reliable delivery of requests through the complex topology of services that comprise a modern, cloud native application. buoyant.io
15. Service Mesh The design Each service will have its own proxy service and all these proxy services together form the “Service Mesh”. All the requests to and from each service will go through the mesh proxies. Proxies are also known as sidecars.
16. Sidecar pattern Service A Business logic Circuit Breaker Rate limiting Tracing Metrics Proxy Service B Business logic Circuit Breaker Rate limiting Tracing Metrics Proxy Injected Network concerns become transparent Service to service communication
17. History of Istio - Envoy proxy (Istio data plane) created by Lyft and open-sourced in 2016. - IBM and Google launch the project in May 2017. - First major version released in July 2018. - Version 1.1 under development.
18. Solution Istio Promises • Focus on business logic and spent less time with common concerns. • No change in the service code. • Central configuration management. • Easy to upgrade • Security
19. Istio does: - Service discovery - Load Balancing & Intelligent Routing - Resiliency: Circuit Breaker & Retry - Rate Limiting - Authentication and Authorization - Service to Service mTLS - Policy enforcement - Observability - Monitoring metrics - Distributed tracing - User authentication and authorization: we still need an IAM (although Istio supports authentication token validation). - Event Driven Asynchronous communication - Service Orchestration Istio does not:
20. Sidecar pattern Challenge 1 Challenge 3 Configuration Service Service Registry / Discovery Circuit Breaker/Retry Rate Limiting API Gateway Authentication & Authorization Monitoring Distributed tracing Event Driven Messaging (Async) Log Aggregation Audit Business Service Foundation Monitoring and Observability Communication Business Values Load Balancing / Intelligent Routing Business Service Business Service
21. Architecture
22. Challenge 1 Challenge 2 Challenge 3
23. Istio building blocks 1/2 Component Description Pilot Responsible for service discovery and for configuring the Envoy sidecar proxies Citadel Automated key and certificate management Mixer Istio-Policy: policy enforcement Istio-Telemetry: gather telemetry data Galley Configuration ingestion for istio components Ingress Gateway manage inbound connection to the service mesh Egress Gateway manage outbound connection from the service mesh Sidecar injector Inside sidecar for enabled namespaces
24. Istio building blocks 1/2 Component Description Prometheus Metrics collection Grafana Monitoring dashboard Jaeger Distributed tracing Kiali Observability dashboard Service Graph Service dependencies
25. Challenge 1 Challenge 2 Challenge 3
26. Demo Application - BookInfo Challenge 1 Challenge 2 Challenge 3 Product Page Application Details Service Review Service Rating Service http://bookinfo.europe-west1-b.innovlabs.io
27. Demo Application - BookInfo
28. Application Deployment istio-init istio-proxy Service container 1 2 3 kubectl describe po/pod-id -n bookinfo-app
29. Envoy proxy initialization - istio-init kubectl logs po/pod-id -c istio-init -n bookinfo-app
30. Dashboard Challenge 1 Challenge 2 Challenge 3 • Monitoring: http://grafana.europe-west1-b.innovlabs.io/ • Observability: http://kiali.europe-west1-b.innovlabs.io/console/overview • Metrics: http://prometheus.europe-west1-b.innovlabs.io/graph • Tracing: http://tracing.europe-west1-b.innovlabs.io/search • Service graph: http://servicegraph.europe-west1-b.innovlabs.io/dotviz
31. Service Discovery Challenge 1 Challenge 2 Challenge 3 Kubernetes provide service discovery, why do I need an extra one ? Istio supports: - HTTP L7 filter - HTTP L7 routing (based on http headers and cookies) - First class HTTP/2 - gRPC support - Fine-grained traffic splitting
32. Ingress Gateway Challenge 1 Challenge 2 Challenge 3 Define access to services from the outside the service mesh: Challenge 1 Challenge 3 Product Page Proxy Details Proxy Ingress Gateway 1 2 3 4 5 6 7 8
33. Ingress Gateway Challenge 1 Challenge 2 Challenge 3 Gateway: defines a load balancer operating at the edge of the mesh receiving incoming (Ingress) or outgoing (Egress) HTTP/TCP connections. VirtualService: defines a set of traffic routing rules to apply when a host is addressed. DestinationRule: defines policies that apply to traffic intended for a service after routing has occurred.
34. Ingress Gateway Challenge 1 Challenge 2 Challenge 3
35. Ingress Gateway Challenge 1 Challenge 2 Challenge 3
36. Ingress Gateway Challenge 1 Challenge 2 Challenge 3 apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: bookinfo-gateway spec: selector: istio: ingressgateway servers: - port: number: 80 name: http protocol: HTTP hosts: - "bookinfo.europe-west1-b.innovlabs.io" apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: bookinfo spec: hosts: - "*" gateways: - bookinfo-gateway http: - match: - uri: exact: /productpage route: - destination: host: productpage port: number: 9080
37. Traffic Management / Splitting Challenge 1 Challenge 3 EDS: Endpoint Discovery Service CDS: Cluster Discovery Service RDS: Route Discovery Service LDS: Listener Discovery Service istioctl proxy-status istioctl proxy-config clusters -n namespace pod-id Product Page Proxy Details Proxy Pilot xDS Sync xDS Sync
38. Traffic Management / Splitting • Achieve affinity by using Cookie, Header, or Source IP. • Each service can have any number of versions (a.k.a. subsets). • Pilot translates high-level rules into low-level configurations and distributes this config to Envoy instances. • Pilot uses three types of configuration resources to manage traffic within its service mesh: Virtual Services, Destination Rules, and Service Entries.
39. Traffic Management / Splitting apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: reviews spec: host: reviews subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2 - name: v3 labels: version: v3 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: reviews spec: hosts: - reviews http: - route: - destination: host: reviews subset: v1 weight: 80 - destination: host: reviews subset: v2 weight: 20
40. A/B testing - Canary release 100% 0%
41. A/B testing - Canary release 100% 0% 80% 20%
42. A/B testing - Canary release 100% 0% 80% 20% 50% 50%
43. A/B testing - Canary release 100% 0% 80% 20% 50% 50% 100% 0%
44. Fault Injection apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: details-service-503 spec: hosts: - details http: - route: - destination: host: details fault: abort: percent: 20 httpStatus: 503 Inject faults to test the resiliency of your application.
45. Circuit Breaker Circuit breaking allows to write applications that limit the impact of failures, latency spikes, and other network effects. apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: details spec: host: details trafficPolicy: connectionPool: tcp: maxConnections: 1 http: http1MaxPendingRequests: 1 maxRequestsPerConnection: 1 outlierDetection: consecutiveErrors: 1 interval: 1s baseEjectionTime: 3m maxEjectionPercent: 100
46. Retry strategy Enable automatic retry when a temporary network issue occurs. Timeout could be also specified. apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: details spec: hosts: - details http: - route: - destination: host: details retries: attempts: 3 perTryTimeout: 2s
47. Egress Gateway
48. Debugging kubectl logs po/POD-ID -c istio-init -n namespace istioctl proxy-status istioctl proxy-config clusters -n namespace POD-ID istioctl proxy-config listeners -n namespace POD-ID
49. Istio Service Mesh - next steps - Mesh Extension: enable managing external services. - Istio CNI: v1.1 https://deploy-preview-2902--preliminary-istio.netlify.com/docs/setup/kuberne tes/istio-cni-install/ - Service Mesh interoperability over cloud providers.
50. Deep dive with istio 1/2 - IBM workshop: https://istio101.gitbook.io/lab/ - Redhat interactive learn: https://learn.openshift.com/servicemesh - Redhat developer demos: https://github.com/redhat-developer-demos/istio-tutorial/ - Istio by example, Ray Tsang: https://github.com/saturnism/istio-by-example-java
51. Deep dive with istio 2/2 Istio in Action, Christian Posta
52. “Istio can (and should) be adopted incrementally.” Christian Posta, Istio in Action
53. Thank you! Any questions ?
54. References - Microservice 4.0 Journey - From Spring NetFlix OSS to Istio Service Mesh and Serverless, Daniel Oh - https://blog.buoyant.io/2017/04/25/whats-a-service-mesh-and-why-do-i-need- one/ - https://istio.io/blog/2017/0.1-using-network-policy/