$30 off During Our Annual Pro Sale. View Details »

MicroservicesOnKube.pdf

Shahid Iqbal
September 18, 2018
680

 MicroservicesOnKube.pdf

Developing microservices based applications is a hot trend in our industry.
Sometimes this decision is based not on trying solving the specific problems that microservices solve but unfortunately a desire to use the latest techniques or, even worse, a desire to adopt trendy new platforms such as Kubernetes and service meshes.

However, let's assume that our audience on the other hand have clearly identified how microservices will benefit them.
This talk introduces a "journey" to microservices and highlights some new problems that are created by adopting such an architecture.

We then step through how we can leverage cloud native technologies such as Kubernetes and the even trendier topic of service meshes such as Istio to try and address some of these new problems.

This talk along with demos will cover many topics in an area which is rapidly evolving however we'll try and focus on the fundamental problems, and demonstrate tools that can help make the experience better or simply easier to manage.

Some knowledge of microservices, containers and Kubernetes is useful but not essential we'll cover the basic topics

Shahid Iqbal

September 18, 2018
Tweet

Transcript

  1. @shahiddev
    Shahid Iqbal | Freelance consultant
    @shahiddev
    Solving microservice
    challenges
    using
    Kubernetes

    View Slide

  2. @shahiddev
    Very brief intro
    Freelance hands-on consultant working on Azure, .NET & Kubernetes
    .NET developer/Architect for 10+ yrs & Microsoft MVP
    Based in the UK and working globally
    Co-organiser of the MK.net meetup in the UK
    @shahiddev on Twitter
    https://www.linkedin.com/in/shahiddev/
    https://blog.headforcloud.com
    https://sessionize.com/shahid-iqbal

    View Slide

  3. @shahiddev
    Agenda
    Technical challenges introduced by moving to microservices
    Addressing those challenges with various cloud native technologies

    View Slide

  4. @shahiddev
    Not covering
    How to decompose your monolith to microservices
    Deep dive into creating containers
    Deep dive into Kubernetes concepts

    View Slide

  5. @shahiddev
    Audience participation

    View Slide

  6. @shahiddev
    Microservices
    “ Microservice architecture, is an approach to application development in
    which a large application is built as a suite of modular components or
    services. ... These services are built around business capabilities and
    independently deployable by fully automated deployment machinery.”
    https://searchmicroservices.techtarget.com/definition/microservices

    View Slide

  7. @shahiddev
    Benefits of microservices architecture
    Independent scaling of services
    Smaller/less risky deployments
    Reduced cognitive load for development teams
    Easier to distribute work across dev teams

    View Slide

  8. @shahiddev
    Starting with a well structured monolith
    Monolith
    admin
    Reporting
    Registration
    Calendar

    View Slide

  9. @shahiddev
    admin Reg
    Calendar
    Reports
    Decompose to microservices
    Monolith admin
    Reporting
    Registration
    Calendar
    microservices

    View Slide

  10. @shahiddev
    Decomposing the monolith
    Monolith
    admin
    Reporting
    Registration
    Calendar

    View Slide

  11. @shahiddev
    Evolution towards microservices architecture
    Monolith
    Registration
    Calendar
    admin
    Reporting

    View Slide

  12. @shahiddev
    Evolution towards microservices architecture
    Monolith
    Registration
    Calendar
    admin
    reporting

    View Slide

  13. @shahiddev
    Evolution towards microservices architecture
    admin
    reporting
    Reg
    Calendar

    View Slide

  14. @shahiddev
    Evolution towards microservices architecture
    admin
    reporting
    Reg
    Calendar
    Front end

    View Slide

  15. @shahiddev
    Evolution towards microservices architecture
    admin
    reporting
    Reg
    Calendar
    Front end

    View Slide

  16. @shahiddev
    New problems we’ve introduced
    Multiple deployments – consistency
    Added infrastructure and environment complexity
    Multiple points of failure/latency
    Debugging applications
    Networking & traffic routing
    Difficulty tracing/logging across those services
    Security concerns – multiple services

    View Slide

  17. @shahiddev
    Containers to the rescue?

    View Slide

  18. @shahiddev
    What are containers?
    Package encapsulating your application and it’s dependencies
    Artefact from your CI system instead of binaries/deployment packages
    Provide isolation and consistent behaviour across environments
    “Lightweight VMs” – quick start up times
    Portable – work anywhere that supports the container runtime

    View Slide

  19. @shahiddev
    Managing multiple containers

    View Slide

  20. @shahiddev
    Container orchestrators to
    the rescue?

    View Slide

  21. @shahiddev
    What is Kubernetes?
    Open source container orchestrator
    Helps you run container based
    applications by managing the containers
    across multiple nodes

    View Slide

  22. @shahiddev
    What is container orchestration?

    View Slide

  23. @shahiddev
    Kubernetes features
    Container orchestration
    Config and secrets management
    Self healing
    Autoscaling
    Service discovery & load balancing
    Basic traffic management (Ingress)
    Many more….

    View Slide

  24. @shahiddev
    Host name based routing using Ingress
    Node 1 Node 2 Node 3
    app1 svc
    app:app1 app:app1
    app:app2
    cluster
    app:app2
    Load
    Balancer
    app2 svc
    Ingress controller
    Public IP
    app1.mydomain.com
    Host: app1.mydomain.com

    View Slide

  25. @shahiddev
    Host name based routing using Ingress
    Node 1 Node 2 Node 3
    app1 svc
    app:app1 app:app1
    app:app2
    cluster
    app:app2
    Load
    Balancer
    app2 svc
    Ingress controller
    Public IP
    app2.mydomain.com
    Host: app2.mydomain.com

    View Slide

  26. @shahiddev
    Scaling our applications in Kubernetes
    2 types of application level scaling
    Horizontal scaling (adding more instances) - HPA
    Vertical scaling (increasing resources allocated to existing instance) – VPA
    Cluster autoscaler (CA)
    Increase number of nodes in your cluster to increase compute capacity
    Virtual Node
    “serverless” scaling of Kubernetes

    View Slide

  27. @shahiddev
    Auto scaling triggers
    Horizontal scaling can be based on metrics from application
    V1 HPA uses CPU/Memory
    V2 HPA (beta) can scale from almost any metric including external metrics
    (e.g. queue depth)
    VPA CPU/Memory usage of application
    Cluster autoscaler based on applications waiting to be scheduled
    due to insufficient cluster resources

    View Slide

  28. @shahiddev
    Virtual Kubelet/Node

    View Slide

  29. @shahiddev
    Virtual Kubelet implementations
    Azure Container Instances
    AWS Fargate
    Hashicorp Nomad
    Service Fabric Mesh
    Azure IoT Edge
    …others

    View Slide

  30. @shahiddev
    Azure Container Instances
    “Serverless” containers
    No infrastructure required
    Per sec billing for running container
    Good for:
    Testing images
    Short lived containers
    Bursting for sudden spikes
    Price reductions -> competitive price vs PaaS/VMs

    View Slide

  31. @shahiddev
    Bursting load using virtual node
    Bursting to ACI to continue scaling
    beyond cluster capacity
    ACI

    View Slide

  32. @shahiddev
    Virtual node public preview

    View Slide

  33. @shahiddev
    DEMO
    VIRTUAL NODE SCALING

    View Slide

  34. @shahiddev
    Scale to zero
    Out of the box Kubernetes unable to scale applications to zero
    instances
    Desirable to scale certain microservices to zero instances
    Message handlers
    “functions” style applications

    View Slide

  35. @shahiddev
    KEDA – Kubernetes Event Driven Autoscaler
    Open source project led by Microsoft and RedHat
    Allows for Kubernetes deployments to be auto scaled based on events
    Scale up from zero -> n instances
    Scale down from n -> zero instances
    Deploy functions (e.g. Azure Functions) and replicate behaviour found in
    managed cloud platforms
    Combine with Virtual node to utilise FaaS alongside traditional microservices using
    consistent platform for managing

    View Slide

  36. @shahiddev
    Operating Kubernetes Cluster
    Managed Kubernetes providers help reduce the operational
    complexity of running Kubernetes yourself
    Google
    Kubernetes
    Engine
    Azure
    Kubernetes
    Service

    View Slide

  37. @shahiddev
    Problem checklist







    Multiple deployments – consistency
    More infrastructure and environment complexity
    Multiple points of failure
    Debugging applications
    Networking & traffic routing
    Tracing/logging across those services
    Security concerns – multiple services

    View Slide

  38. @shahiddev
    Tooling to the rescue?
    Azure Dev Spaces
    Kustomize

    View Slide

  39. @shahiddev
    From code to Kubernetes
    Write code
    Push code to
    repository
    Build
    containers
    Push
    container to
    registry
    Create/update
    deployment
    “package”
    Deploy
    “package” to
    K8s
    Continuous Integration Deployment
    Local development

    View Slide

  40. @shahiddev
    Working with dependent services
    Frontend Service A
    Service B
    1. Mock/stub out dependent services
    2. Run all dependent services on my
    local machine
    3. Run everything on remote cluster
    4. Run dependent services on remote
    cluster

    View Slide

  41. @shahiddev
    AKS cluster
    Azure Dev Spaces
    Frontend Service A Service B
    laptop
    app.mydomain.com
    Service A
    V2
    Shahid.s.app.mydomain.com

    View Slide

  42. @shahiddev
    Shahid dev space
    Default dev space
    Azure Dev Spaces
    Frontend Service A Service B
    app.mydomain.com
    Service A
    V2
    shahid.s.app.mydomain.com
    AKS cluster

    View Slide

  43. @shahiddev
    Azure Dev Spaces
    Works with VS Code and Visual Studio
    First class support for
    .NET Core, NodeJS & Java
    Limited support for other languages
    Use it to dev/test applications on AKS even if you deploy prod to other
    Kubernetes clusters
    GA as of May 6th 2019
    https://docs.microsoft.com/en-us/azure/dev-spaces/

    View Slide

  44. @shahiddev
    DEMO
    AZURE DEV SPACES

    View Slide

  45. @shahiddev
    Problem checklist







    Multiple deployments – consistency
    More infrastructure and environment complexity
    Multiple points of failure
    Debugging applications
    Networking & traffic routing
    Tracing/logging across those services
    Security concerns – multiple services

    View Slide

  46. @shahiddev
    Service mesh to the rescue?

    View Slide

  47. @shahiddev
    What is a Service mesh?
    “A service mesh is a dedicated infrastructure layer for handling
    service-to-service communication.
    It’s responsible for the reliable delivery of requests through the
    complex topology of services that comprise a modern, cloud native
    application.”
    https://blog.buoyant.io/2017/04/25/whats-a-service-mesh-and-why-do-i-need-one/

    View Slide

  48. @shahiddev
    What is Istio?
    Open source project being led by Lyft, IBM & Google
    Runs on top of Kubernetes (and other platforms) to extend
    capabilities
    Provides:
    • Network error handling (retry, circuit breakers, timeouts)
    • Rate limiting
    • Traffic shifting
    • Fault injection
    • Service to service communication security & policies
    • Distributed tracing of requests

    View Slide

  49. @shahiddev
    Istio architecture
    Data plane
    Management plane

    View Slide

  50. @shahiddev
    Deploying Istio to your cluster and applications
    Cluster
    • Istio can be installed in Kubernetes using Helm
    • Using a number of Custom Resource Definitions (CRDs) to extend
    behaviour
    Application
    • Envoy proxy can be automatically injected into your applications
    during deployment
    • Generate updated manifest files using Istioctl

    View Slide

  51. @shahiddev
    Securing service to service communications
    Products
    Product
    reviews
    Product
    ratings
    Product
    details

    View Slide

  52. @shahiddev
    Securing service to service communications
    Products
    Product
    reviews
    Product
    ratings
    Product
    details

    View Slide

  53. @shahiddev
    Traffic shifting
    Products
    Product
    reviews
    Product
    ratings
    Product
    details

    View Slide

  54. @shahiddev
    Traffic shifting
    Products
    Product
    reviews
    V1
    Product
    ratings
    Product
    details
    Product
    reviews
    V2
    User=Jason
    Jason

    View Slide

  55. @shahiddev
    Canary deployments
    Products
    Product
    reviews
    V1
    Product
    ratings
    Product
    details
    Product
    reviews
    V2
    10%
    90%

    View Slide

  56. @shahiddev
    Observability and metrics
    • Metrics collected in Prometheus
    • Built in Grafana dashboards to display service level metrics
    • Distributed tracing using Jaegar

    View Slide

  57. @shahiddev
    Demo Application
    Products
    Product
    reviews
    V1
    Product
    ratings
    Product
    details
    Product
    reviews
    V2
    No stars

    View Slide

  58. @shahiddev
    DEMO
    ISTIO

    View Slide

  59. @shahiddev
    Problem checklist







    Multiple deployments – consistency
    More infrastructure and environment complexity
    Multiple points of failure
    Debugging applications
    Networking & traffic routing
    Tracing/logging across those services
    Security concerns – multiple services

    View Slide

  60. @shahiddev
    Summary
    Cloud native technologies can help alleviate technical complexity
    caused by adopting microservices
    Complexity is rarely removed, just shifted (and hopefully becomes someone else's problem)
    Don’t be peer pressured into adopting microservices
    Developer tooling for Kubernetes is an area of rapid innovation
    Service meshes can help with service to service communications
    But you don’t always need them!

    View Slide

  61. @shahiddev
    Where can I go to learn more?
    http://www.katacoda.com
    https://katacoda.com/courses/istio/
    https://bit.ly/k8s-ndc
    https://istio.io

    View Slide

  62. @shahiddev
    Shahid Iqbal | Freelance consultant
    @shahiddev
    Thank you!
    Slides: https://bit.ly/k8s-microservices
    Video: https://bit.ly/k8s-microservices-video
    @shahiddev on Twitter
    https://www.linkedin.com/in/shahiddev/
    https://blog.headforcloud.com

    View Slide