Service Meshes: Powering the next wave of microservice architectures

Microservices and containers have transformed application design and deployment patterns. Modern cloud native architectures - which underpin many of the world's groundbreaking tech companies such as Uber, Netflix and Airbnb - afford unparalleled levels of agility and scale but are not without trade-offs. In applications comprising hundreds of services (and thousands of service instances), concerns such as security, operability and observability pose significant challenges. Supporting compensating capabilities such as circuit breakers, retry policies and service discovery in each microservice adds undesirable code bloat and impacts our ability to choose the best language for the problem at hand should the required libraries not be available.

In this session, we will take an in-depth look at the service mesh pattern; the benefits that a decentralized microservice management approach brings; the best practices that have evolved; and most importantly what you need to know to effectively leverage a service mesh in your architecture:

- How to determine the suitability of a service mesh for your application context. Don't just drink the cloud native Kool-Aid!
- Disambiguate the overlapping responsibilities of an API gateway and service mesh in a modern architecture and demonstrate how they can be used in harmony.
- How a service mesh enables choosing the best tool or language for a given service without being constrained by the availability of libraries for each platform. Effectively it's a weight loss plan for your microservices.
- What techniques to apply when transitioning to a service mesh based architecture.

Additionally, we will take a hands-on look at what is involved to build and manage a microservice architecture leveraging Kubernetes and Istio, a leading open source service mesh. By the end of the session, you will not only understand the concepts underpinning the service mesh pattern but also have the knowledge to put them into practice.

Transcript

1. @robdcrowley | robdcrowley Service Meshes Powering the next wave of microservice architectures
2. So who has heard of the Service Mesh pattern? @robdcrowley
3. Is anyone using a Service Mesh in production? @robdcrowley
4. Goals for the session ▪ Explore the motivation behind the Service Mesh Pattern and the category of problems it helps to solve. ▪ Contrast Service Meshes with API Gateways and our old favourite the Enterprise Service Bus. ▪ Investigate some of the advanced capabilities Service Meshes provide to enable low friction releases at scale. ▪ Starting a conversation in the Sydney technical community about the potential value of Service Meshes. @robdcrowley
5. A smart person once told me that every good presentation has a story… @robdcrowley
6. Service A Service B A ♥ story of when service A met service B… Host A @robdcrowley
7. Service A Service B As part of a Digital Transformation Service B is rehosted Host A Host B @robdcrowley
8. Distributed computing is easy, right? @robdcrowley
9. Service A Service B Long distance relationships are complicated Host A Host B @robdcrowley
10. The 8 Fallacies of Distributed Computing highlight common false assumptions we programmers make when starting out with distributed systems @robdcrowley
11. Service A Service B Long distance relationships are complicated Host A Host B Flow Control Flow Control Networking Networking @robdcrowley
12. Hmmm, maybe this distributed computing thing is a little harder than we thought @robdcrowley
13. Traffic Management Observability Reliability Wait, there’s more… @robdcrowley
14. Traffic Management ▪ Load balancing (bonus points for being latency based) ▪ Service discovery (DNS alone is often insufficient) ▪ Access control ▪ Per request routing ▪ Shadowing ▪ Fault injection @robdcrowley
15. Observability ▪ Success rates ▪ Latency logs ▪ Request volumes ▪ Distributed tracing (e.g. Open Tracing) @robdcrowley
16. Reliability (not anti-fragility! ) ▪ Health checks ▪ Circuit breakers ▪ Retry policies (your services are idempotent right?) ▪ Timeouts @robdcrowley
17. Hmmm, maybe this distributed computing thing is a lot harder than we thought @robdcrowley
18. Service A Service B Making long distance relationships work Host A Host B Networking Networking Flow Control @robdcrowley Flow Control
19. TCP/IP as applied to OSI 7. Application 6. Presentation 5. Session 4. Transport 3. Network 2. Data Link 1. Physical Network Access Internet (IP) Transport (TCP) Application (HTTP) OSI TCP/IP @robdcrowley
20. As application requirements became more sophisticated we again looked to abstract these capabilities @robdcrowley
21. Centralised management of your services Enterprise Service Bus Service E Service H Service F Service G Service A Service D Service B Service C @robdcrowley
22. centralised architecture == single point of failure @robdcrowley
23. Over time these centralised architectures tend to become complex and atrophy change @robdcrowley
24. Continuous Delivery Microservices Cloud Native DevOps Containers @robdcrowley
25. Life in a microservices world… @robdcrowley
26. Smart endpoints, dumb pipes @robdcrowley
27. Service A Service B Very long distance relationships are even worse Host A Host B Networking Networking Service Discovery Flow Control Flow Control Circuit Breakers Service Discovery Circuit Breakers @robdcrowley
28. Wait, I thought one of the benefits of microservices is the ability to match the language to the specific problem domain? @robdcrowley
29. Sidecar Proxy Service A Making very long distance relationships work Host A Networking Flow Control Service B Host B Networking Flow Control Sidecar Proxy Service Discovery Circuit Breakers Service Discovery Circuit Breakers @robdcrowley
30. Life in a better microservices world… @robdcrowley
31. A service mesh is a dedicated infrastructure layer for handling service-to-service communication. It’s responsible for the reliable delivery of requests through the complex topology of services that comprise a modern, cloud native application. In practice, the service mesh is typically implemented as an array of lightweight network proxies that are deployed alongside application code, without the application needing to be aware. - William Morgan @robdcrowley
32. The goal of service meshes is not to introduce new functionality, but rather to enable a shift in where functionality is located @robdcrowley
33. Service meshes comprise both a data plane and a control plane @robdcrowley
34. Decentralised management of your services Service Mesh Control Plane @robdcrowley
35. Decentralised management of your services Service Mesh Control Plane @robdcrowley
36. The control plane provides the same authoritative source of policies and routing decisions as with an ESB, without being on the critical path of every request @robdcrowley
37. @robdcrowley
38. Istio @robdcrowley
39. A high level view of Istio Service A Sidecar @robdcrowley Service B Sidecar HTTP/1.1, HTTP/2, gRPC or TCP . mTLS is optional Pilot Citadel Mixer
40. A high level view of Istio Service A Sidecar @robdcrowley Service B Sidecar HTTP/1.1, HTTP/2, gRPC or TCP . mTLS is optional Pilot Citadel Mixer
41. A high level view of Istio Service A Sidecar @robdcrowley Service B Sidecar HTTP/1.1, HTTP/2, gRPC or TCP . mTLS is optional Pilot Citadel Mixer Policy checks & telemetry
42. A high level view of Istio Service A Sidecar @robdcrowley Service B Sidecar HTTP/1.1, HTTP/2, gRPC or TCP . mTLS is optional Pilot Citadel Mixer
43. Istio in Action @robdcrowley
44. Family dynamics Service A @robdcrowley Service B Service C V2 Service C V3 Service C V1 Service D
45. Testing in production is not just for hipsters… …the trick is doing it safely
47. Traffic Mirroring @robdcrowley
48. Service meshes simplify some advanced use cases Service A Service B V1 Service B V2 Envoy Envoy Envoy @robdcrowley
49. Canary Releases @robdcrowley
50. @robdcrowley Service A Service B V1 Service B V2 Envoy Envoy Envoy Service meshes simplify some advanced use cases Pilot Push traffic routing rules to sidecar proxy
51. Circuit Breakers @robdcrowley
52. @robdcrowley Service A Envoy Service meshes simplify some advanced use cases Service C V1 Envoy Service C V2 Envoy Service C V3 Envoy
53. @robdcrowley Service A Envoy Service meshes simplify some advanced use cases Eject host from connection pool Service C V1 Envoy Service C V2 Envoy Service C V3 Envoy X
54. Chaos Engineering @robdcrowley
55. @robdcrowley Service A Envoy Service C V1 Envoy Service C V2 Envoy Service D Envoy
56. @robdcrowley Service A Envoy Service C V1 Envoy Service C V2 Envoy X Service D Envoy Inject faults for 25% traffic to D
57. Per Request Overrides @robdcrowley
58. @robdcrowley Service meshes decouple traffic management from infrastructure scaling
59. Service meshes are typically supported by a platform team within an enterprise @robdcrowley
60. Pop Quiz: So do application service developers even need to be aware of the service mesh? @robdcrowley
61. @robdcrowley
62. So, how should you approach introducing a service mesh into your architecture? @robdcrowley
63. @robdcrowley
64. Start with a single capability, say instrumentation, and support that via the service mesh @robdcrowley
65. Takeaways @robdcrowley
66. A service mesh is a dedicated infrastructure layer for making service-to-service communication safe @robdcrowley
67. If the operability of your services is posing a significant challenge, then a Service Mesh will help @robdcrowley
68. Service Meshes afford gradual adoption. Start with single use case. @robdcrowley
69. Questions @robdcrowley
70. Thank You! @robdcrowley | robdcrowley
71. Resources ▪ 8 Fallacies of Distributed Computing: ▪ Overview: https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing ▪ Service Mesh: ▪ Overview: https://buoyant.io/2017/04/25/whats-a-service-mesh-and-why-do-i-need-one/ ▪ InfoQ Panel: https://www.infoq.com/articles/vp-microservices-communication-governance-using- service-mesh ▪ InfoQ Podcast: https://www.infoq.com/podcasts/Oliver-Gould-microservices-linkerd-service-mesh ▪ Istio: ▪ Overview: https://istio.io/docs/concepts/what-is-istio/ ▪ Kelsey Hightower Velocity Talk: https://www.youtube.com/watch?v=6BYq6hNhceI ▪ Istio Demo Application: https://github.com/robcrowley/istio-demo @robdcrowley
72. Slides are available at bit.ly/ndcsydney-mesh @robdcrowley
73. Appendices @robdcrowley
74. Linkerd as daemon set on Kubernetes Linkerd Service A Service B Service C Service E Service D Service F Linkerd Service G Service H Service I Service K Service J Service L Host A Host B Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod