Service Meshes: Powering the next wave of microservice architectures

Microservices and containers have transformed application design and deployment patterns. Modern cloud native architectures - which underpin many of the world's groundbreaking tech companies such as Uber, Netflix and Airbnb - afford unparalleled levels of agility and scale but are not without trade-offs. In applications comprising hundreds of services (and thousands of service instances), concerns such as security, operability and observability pose significant challenges. Supporting compensating capabilities such as circuit breakers, retry policies and service discovery in each microservice adds undesirable code bloat and impacts our ability to choose the best language for the problem at hand should the required libraries not be available.

In this session, we will take an in-depth look at the service mesh pattern; the benefits that a decentralized microservice management approach brings; the best practices that have evolved; and most importantly what you need to know to effectively leverage a service mesh in your architecture:

- How to determine the suitability of a service mesh for your application context. Don't just drink the cloud native Kool-Aid!
- Disambiguate the overlapping responsibilities of an API gateway and service mesh in a modern architecture and demonstrate how they can be used in harmony.
- How a service mesh enables choosing the best tool or language for a given service without being constrained by the availability of libraries for each platform. Effectively it's a weight loss plan for your microservices.
- What techniques to apply when transitioning to a service mesh based architecture.

Additionally, we will take a hands-on look at what is involved to build and manage a microservice architecture leveraging Kubernetes and Istio, a leading open source service mesh. By the end of the session, you will not only understand the concepts underpinning the service mesh pattern but also have the knowledge to put them into practice.

Transcript

1. @robdcrowley | robdcrowley
   Service Meshes
   Powering the next wave of microservice architectures
   

   View Slide

2. So who has heard of the
   Service Mesh pattern?
   @robdcrowley
   

   View Slide

3. Is anyone using a Service
   Mesh in production?
   @robdcrowley
   

   View Slide

4. Goals for the session
   ▪ Explore the motivation behind the Service Mesh Pattern
   and the category of problems it helps to solve.
   ▪ Contrast Service Meshes with API Gateways and our old
   favourite the Enterprise Service Bus.
   ▪ Investigate some of the advanced capabilities Service
   Meshes provide to enable low friction releases at scale.
   ▪ Starting a conversation in the Sydney technical
   community about the potential value of Service Meshes.
   @robdcrowley
   

   View Slide

5. A smart person once told me that every good
   presentation has a story…
   @robdcrowley
   

   View Slide

6. Service A Service B
   A ♥ story of when service A met service B…
   Host A
   @robdcrowley
   

   View Slide

7. Service A Service B
   As part of a Digital Transformation Service B is rehosted
   Host A Host B
   @robdcrowley
   

   View Slide

8. Distributed computing is easy, right?
   @robdcrowley
   

   View Slide

9. Service A Service B
   Long distance relationships are complicated
   Host A Host B
   @robdcrowley
   

   View Slide

10. The 8 Fallacies of Distributed Computing highlight
    common false assumptions we programmers
    make when starting out with distributed systems
    @robdcrowley
    

    View Slide

11. Service A Service B
    Long distance relationships are complicated
    Host A Host B
    Flow Control Flow Control
    Networking Networking
    @robdcrowley
    

    View Slide

12. Hmmm, maybe this distributed computing thing
    is a little harder than we thought
    @robdcrowley
    

    View Slide

13. Traffic
    Management
    Observability
    Reliability
    Wait, there’s more…
    @robdcrowley
    

    View Slide

14. Traffic Management
    ▪ Load balancing (bonus points for being latency based)
    ▪ Service discovery (DNS alone is often insufficient)
    ▪ Access control
    ▪ Per request routing
    ▪ Shadowing
    ▪ Fault injection
    @robdcrowley
    

    View Slide

15. Observability
    ▪ Success rates
    ▪ Latency logs
    ▪ Request volumes
    ▪ Distributed tracing (e.g. Open Tracing)
    @robdcrowley
    

    View Slide

16. Reliability (not anti-fragility! )
    ▪ Health checks
    ▪ Circuit breakers
    ▪ Retry policies (your services are idempotent right?)
    ▪ Timeouts
    @robdcrowley
    

    View Slide

17. Hmmm, maybe this distributed computing thing
    is a lot harder than we thought
    @robdcrowley
    

    View Slide

18. Service A Service B
    Making long distance relationships work
    Host A Host B
    Networking Networking
    Flow Control
    @robdcrowley
    Flow Control
    

    View Slide

19. TCP/IP as applied to OSI
    7. Application
    6. Presentation
    5. Session
    4. Transport
    3. Network
    2. Data Link
    1. Physical
    Network Access
    Internet
    (IP)
    Transport
    (TCP)
    Application
    (HTTP)
    OSI
    TCP/IP
    @robdcrowley
    

    View Slide

20. As application requirements became more
    sophisticated we again looked to abstract
    these capabilities
    @robdcrowley
    

    View Slide

21. Centralised management of your services
    Enterprise Service Bus
    Service E Service H
    Service F Service G
    Service A Service D
    Service B Service C
    @robdcrowley
    

    View Slide

22. centralised architecture == single point of failure
    @robdcrowley
    

    View Slide

23. Over time these centralised architectures tend
    to become complex and atrophy change
    @robdcrowley
    

    View Slide

24. Continuous
    Delivery
    Microservices
    Cloud
    Native
    DevOps
    Containers
    @robdcrowley
    

    View Slide

25. Life in a microservices world…
    @robdcrowley
    

    View Slide

26. Smart endpoints, dumb pipes
    @robdcrowley
    

    View Slide

27. Service A Service B
    Very long distance relationships are even worse
    Host A Host B
    Networking Networking
    Service Discovery
    Flow Control Flow Control
    Circuit Breakers
    Service Discovery
    Circuit Breakers
    @robdcrowley
    

    View Slide

28. Wait, I thought one of the benefits of microservices
    is the ability to match the language to the specific
    problem domain?
    @robdcrowley
    

    View Slide

29. Sidecar Proxy
    Service A
    Making very long distance relationships work
    Host A
    Networking
    Flow Control
    Service B
    Host B
    Networking
    Flow Control
    Sidecar Proxy
    Service Discovery
    Circuit Breakers
    Service Discovery
    Circuit Breakers
    @robdcrowley
    

    View Slide

30. Life in a better microservices world…
    @robdcrowley
    

    View Slide

31. A service mesh is a dedicated infrastructure layer for
    handling service-to-service communication. It’s responsible
    for the reliable delivery of requests through the complex
    topology of services that comprise a modern, cloud native
    application. In practice, the service mesh is typically
    implemented as an array of lightweight network proxies
    that are deployed alongside application code, without the
    application needing to be aware.
    - William Morgan
    @robdcrowley
    

    View Slide

32. The goal of service meshes is not to introduce
    new functionality, but rather to enable a shift in
    where functionality is located
    @robdcrowley
    

    View Slide

33. Service meshes comprise both a data plane
    and a control plane
    @robdcrowley
    

    View Slide

34. Decentralised management of your services
    Service Mesh Control Plane
    @robdcrowley
    

    View Slide

35. Decentralised management of your services
    Service Mesh Control Plane
    @robdcrowley
    

    View Slide

36. The control plane provides the same authoritative
    source of policies and routing decisions as with
    an ESB, without being on the critical path of
    every request
    @robdcrowley
    

    View Slide

37. @robdcrowley
    

    View Slide

38. Istio
    @robdcrowley
    

    View Slide

39. A high level view of Istio
    Service A
    Sidecar
    @robdcrowley
    Service B
    Sidecar
    HTTP/1.1, HTTP/2, gRPC or TCP
    .
    mTLS is optional
    Pilot Citadel
    Mixer
    

    View Slide

40. A high level view of Istio
    Service A
    Sidecar
    @robdcrowley
    Service B
    Sidecar
    HTTP/1.1, HTTP/2, gRPC or TCP
    .
    mTLS is optional
    Pilot Citadel
    Mixer
    

    View Slide

41. A high level view of Istio
    Service A
    Sidecar
    @robdcrowley
    Service B
    Sidecar
    HTTP/1.1, HTTP/2, gRPC or TCP
    .
    mTLS is optional
    Pilot Citadel
    Mixer
    Policy checks & telemetry
    

    View Slide

42. A high level view of Istio
    Service A
    Sidecar
    @robdcrowley
    Service B
    Sidecar
    HTTP/1.1, HTTP/2, gRPC or TCP
    .
    mTLS is optional
    Pilot Citadel
    Mixer
    

    View Slide

43. Istio in Action
    @robdcrowley
    

    View Slide

44. Family dynamics
    Service A
    @robdcrowley
    Service B
    Service C
    V2
    Service C
    V3
    Service C
    V1
    Service D
    

    View Slide

45. Testing in production is not just for hipsters…
    …the trick is doing it safely
    

    View Slide

46. View Slide

47. Traffic Mirroring
    @robdcrowley
    

    View Slide

48. Service meshes simplify some advanced use cases
    Service A
    Service B
    V1
    Service B
    V2
    Envoy
    Envoy
    Envoy
    @robdcrowley
    

    View Slide

49. Canary Releases
    @robdcrowley
    

    View Slide

50. @robdcrowley
    Service A
    Service B
    V1
    Service B
    V2
    Envoy
    Envoy
    Envoy
    Service meshes simplify some advanced use cases
    Pilot
    Push traffic routing rules to sidecar proxy
    

    View Slide

51. Circuit Breakers
    @robdcrowley
    

    View Slide

52. @robdcrowley
    Service A
    Envoy
    Service meshes simplify some advanced use cases
    Service C
    V1
    Envoy
    Service C
    V2
    Envoy
    Service C
    V3
    Envoy
    

    View Slide

53. @robdcrowley
    Service A
    Envoy
    Service meshes simplify some advanced use cases
    Eject host from
    connection pool
    Service C
    V1
    Envoy
    Service C
    V2
    Envoy
    Service C
    V3
    Envoy
    X
    

    View Slide

54. Chaos Engineering
    @robdcrowley
    

    View Slide

55. @robdcrowley
    Service A
    Envoy
    Service C
    V1
    Envoy
    Service C
    V2
    Envoy
    Service D
    Envoy
    

    View Slide

56. @robdcrowley
    Service A
    Envoy
    Service C
    V1
    Envoy
    Service C
    V2
    Envoy
    X
    Service D
    Envoy
    Inject faults for 25% traffic to D
    

    View Slide

57. Per Request Overrides
    @robdcrowley
    

    View Slide

58. @robdcrowley
    Service meshes decouple traffic management
    from infrastructure scaling
    

    View Slide

59. Service meshes are typically supported by a
    platform team within an enterprise
    @robdcrowley
    

    View Slide

60. Pop Quiz: So do application service developers
    even need to be aware of the service mesh?
    @robdcrowley
    

    View Slide

61. @robdcrowley
    

    View Slide

62. So, how should you approach
    introducing a service mesh into your
    architecture?
    @robdcrowley
    

    View Slide

63. @robdcrowley
    

    View Slide

64. Start with a single capability, say instrumentation,
    and support that via the service mesh
    @robdcrowley
    

    View Slide

65. Takeaways
    @robdcrowley
    

    View Slide

66. A service mesh is a dedicated infrastructure layer
    for making service-to-service communication safe
    @robdcrowley
    

    View Slide

67. If the operability of your services is posing a
    significant challenge, then a Service Mesh will help
    @robdcrowley
    

    View Slide

68. Service Meshes afford gradual adoption.
    Start with single use case.
    @robdcrowley
    

    View Slide

69. Questions
    @robdcrowley
    

    View Slide

70. Thank You!
    @robdcrowley | robdcrowley
    

    View Slide

71. Resources
    ▪ 8 Fallacies of Distributed Computing:
    ▪ Overview: https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing
    ▪ Service Mesh:
    ▪ Overview: https://buoyant.io/2017/04/25/whats-a-service-mesh-and-why-do-i-need-one/
    ▪ InfoQ Panel: https://www.infoq.com/articles/vp-microservices-communication-governance-using-
    service-mesh
    ▪ InfoQ Podcast: https://www.infoq.com/podcasts/Oliver-Gould-microservices-linkerd-service-mesh
    ▪ Istio:
    ▪ Overview: https://istio.io/docs/concepts/what-is-istio/
    ▪ Kelsey Hightower Velocity Talk: https://www.youtube.com/watch?v=6BYq6hNhceI
    ▪ Istio Demo Application: https://github.com/robcrowley/istio-demo
    @robdcrowley
    

    View Slide

72. Slides are available at
    bit.ly/ndcsydney-mesh
    @robdcrowley
    

    View Slide

73. Appendices
    @robdcrowley
    

    View Slide

74. Linkerd as daemon set on Kubernetes
    Linkerd
    Service A
    Service B
    Service C
    Service E
    Service D
    Service F
    Linkerd
    Service G
    Service H
    Service I
    Service K
    Service J
    Service L
    Host A Host B
    Pod Pod
    Pod Pod
    Pod
    Pod
    Pod
    Pod
    Pod
    Pod
    Pod Pod
    

    View Slide