Service Mesh Explained (Istio / Envoy & AWS App Mesh)

A service mesh adds new qualities to your container orchestration. It connects, secures, observes and controls service invocations in a language agnostic way that doesn't require any code changes. In this session I will explain service meshes from the ground up. Then we explore the newly announced AWS App Mesh, analyze its benefits, delineate it to open-source solutions, and describe how it improves your infrastructure by providing several cross-cutting concerns.

Transcript

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Service Mesh Explained Frank Munz Senior Technical Evangelist Amazon Web Services @frankmunz
2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. About me • Software Architect / DevOps Engineer • Technical Evangelist @ AWS • Published an AWS book • Containers, serverless and a sprinkle of ML & big / fast data @frankmunz
3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 10+ Years Back in Time: Enterprise Service Bus + SOA ESB = Service Virtualization Layer • Reduces complexity #cx: squared -> linear with ESB • VETO pattern = Validate, Enrich, Transform, Operate • CCC = location transparency, throttling, monitoring, security, audit
4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Splitting the Monolith – A. Cockroft @ AWS https://youtu.be/aBcG57Gw9k0
5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Enterprise SW Modernization -> Microservices Building Blocks / Technical Architecture • Containers • Serverless / AWS Lambda • Other cloud services • Do NOT stuff everything into a container! • AWS API Gateway, Kinesis, Aurora, Dynamo DB, etc. -> Container / K8s will not make other cloud services redundant
6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Challenges of Containers at Scale • More transient • More distributed and complex • Networking • Scheduling / Resource Management • Not virtualized, but isolated: containers share Linux kernel -> Tooling and orchestration required
7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ECS Easiest way to deploy and manage containers at scale Integration with entire AWS platform ALB, Auto Scaling, Batch, Elastic Beanstalk, CloudFormation, CloudTrail, CloudWatch Events, CloudWatch Logs, CloudWatch Metrics, ECR, EC2 Spot, IAM, NLB, Parameter Store, and VPC Scales to support clusters of any size Service integrations (like ALB and NLB) are at container level 1 2 3
8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes entered the stage
9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. … and devops love Kubernetes
10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Open source container management platform Helps you run containers at scale Gives you primitives for building modern applications What is Kubernetes (K8s)?
11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Run Kubernetes for me.”
12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Availability Zone 1 Availability Zone 2 Availability Zone 3 Kubectl EKS Architecture
13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EKS is Kubernetes Certified + Heptio IAM Authenticator (open sourced) VPC Networking (open sourced)
14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes Ressources (incomplete list) Ressource Pod Basic K8s unit, co-located containers Namespace Non-overlapping group of ressources Replica Set Keeps pod replicas running Service Exposes pod at single stable IP Deployment Rolling update of pods Ingress Expose service with static IP to external client Admission Controller Run code after API request, e.g. inject sidecar
15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. OSS Hystrix: code changes required Service Mesh: decentral, language agnostic, dumb endpoints https://www.infoq.com/articles/microservices-post-kubernetes Shift in Infrastructure Logic ESB: clustered monolith
16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Istio Service Mesh • Connect, secure, and observe services • Shift in where functionality is located • Istio control plane • Data plane = set of all proxies • Envoy proxy as sidecar in K8s pod • Automatic / manual injection of proxy
17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Istio Service Mesh with Envoy Proxy Add a 5s delay to 10% of all requests
18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. User Based Routing Traffic Shifting
19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Envoy Proxy • Level 7 proxy • HTTP, HTTP/2, gRPC, AWS Dynamo DB, MongoDB • C++11 code base , only 8 MB (statically linked) • No language or framework dependencies • No code changes
20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Service Mesh Cross Cutting Concerns such as retries, timeouts, circuit breaking, fault injection, client-side load balancing, service discovery, security, metrics-collection, A/B deployments, and traffic mirroring/ routing
21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Service Mesh But Docker / Kubernetes can do rolling updates! Yes, but Istio sparates traffic flow from replica deployment
22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. A bath tub full of cold water ? K8s rolling update 25% 1 pod at a time … or just wetten your feet? Service Mesh 3% Traffic routing ! ! ! " # ! $❄$❄$❄ Fancy a Swim in the Arctic Sea ? Blue / Green 100% All services at once Microservices Update Strategies
23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Install Istio with Helm $ helm install --wait --name istio \ --namespace istio-system \ install/kubernetes/helm/istio \ --set grafana.enabled=true \ --set tracing.enabled=true \ --set servicegraph.enabled=true $ # K8s label turn on automatic sidecar injection $ kubectl label namespace default istioinjection=enabled
25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Grafana
26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Jaeger
27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Servicegraph
28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customers
29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Snap @AWS Summit in New York 2018 https://youtu.be/mCVdcz01Z-g?t=2052
30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS App Mesh
31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. App Mesh App Mesh is PREVIEW as of 03/2019 Uses on Envoy proxy, but implements own control plane • Envoy is a battle proved service mesh proxy • App Mesh works across clusters and across EKS and ECS (also with Fargate) • Will support bringing your own Envoy images and Istio Mixer in the future.
32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. App Mesh Ressources App Mesh Ressource Virtual Node Logical pointer to a particular task group (ECS service / Kubernetes deployment). Uses DNS service discovery name for your task group Virtual Router Handles traffic for one or more service names within your mesh. Routes Associated with a virtual router, and it directs traffic that matches a service name prefix to one or more virtual nodes.
33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. App Mesh Architecture
34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GA / Post GA Roadmap is Public https://github.com/awslabs/aws-app-mesh-examples
35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Summary • Architect wisely. • Running K8s is hard, run it in the cloud & use K8s service like AWS EKS • Service Meshes complement K8s • AWS EKS is unforked, upstream K8s. • You can install open-source Istio with Envoy as a helm chart on EKS or • AWS App Mesh (preview) uses Envoy
36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. App Mesh References Product overview https://aws.amazon.com/app-mesh Documentation https://docs.aws.amazon.com/app-mesh/index.html Examples https://github.com/awslabs/aws-app-mesh-examples Issues & Roadmap https://github.com/awslabs/aws-app-mesh-examples/issues
37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. General References Introduction to modern network load balancing and proxying https://blog.envoyproxy.io/introduction-to-modern-network-load-balancing-and-proxying- a57f6ff80236 InfoQ: Microservices in a Post-Kubernetes Era https://www.infoq.com/articles/microservices-post-kubernetes Getting Started with Istio on Amazon EKS https://aws.amazon.com/blogs/opensource/getting-started-istio-eks/ EKS Workshop (K8s, Helm, CI/CD, Grafana, Kabana) https://eksworkshop.com/introduction/ Istio Book https://www.manning.com/books/istio-in-action AWS EKS Documentation https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html Envoy with DynamoDB https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/dynamo
38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Open-source Istio and Enovy on AWS EKS Video https://www.youtube.com/watch?v=fDmJf9kWFws
39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Please rate this session:
40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Tomorrow: EKS Workshop
41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SUMMIT Cape Town, 11 July 2019, CTICC
42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Register for free:
43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank You! @frankmunz