Tame your Microservices with Helm and Software Release Trains

The deployment of apps in Kubernetes is simplified with the usage of Helm, but when it comes to deploying microservice architecture things can go a bit more wild as we have to manage multiple applications on potentially multiple environments. We'll discuss the different patterns for storing the charts on Git, either as mono or multi repos. We'll go through the ways to simplify and keep DRY our charts, and lastly we'll go review the different deployment patterns with Helm, either together as software release trains, or individually as with CI/CD.

Transcript

1. View Slide

2. Tame your Microservices
   with Helm and Software Release Trains
   Dimitris Kapanidis
   Harbur Cloud Solutions
   

   View Slide

3. About Me
   Founder at Harbur Cloud Solutions
   Member:
   - Docker Captains
   - Google Developer Experts
   Co-organizer:
   - Docker Barcelona Meetup
   - Kubernetes Barcelona Meetup
   3
   

   View Slide

4. Harbur Cloud Solutions
   We are a cloud native consulting
   ● DevOps Consulting Services
   ● Cloud Native Solutions
   ● Courses & Workshops
   About Us
   4
   

   View Slide

5. About Us
   5
   

   View Slide

6. Tame your Microservices > Content
   6
   Microservice Architecture
   Why GitOps?
   CI/CD with Helm
   Repository Patterns (& Software Release Trains with Helm)
   Development Styles
   Demo! (with GKE + GitHub actions)
   

   View Slide

7. Software Release Trains
   7
   App1
   v1.0.0
   App2
   v1.5.0
   App3
   v2.0.0
   App4
   v1.1.4
   SRT
   v1.0.0
   

   View Slide

8. 8
   Software Release Trains
   SRT
   v1.0.0
   PRO
   TEST
   DEV PRE
   

   View Slide

9. Everything is connected
   Git
   Cloud Native
   Containers
   Microservices
   Helm
   DevOps
   9
   

   View Slide

10. Microservice Architecture > Monolith
    10
    Traffic
    Data Center 1
    Data Center 2
    Monolith
    Monolith
    Monolith
    Monolith
    

    View Slide

11. Microservice Architecture > Microservices
    11
    Traffic
    Cluster 2
    Cluster 1
    Data Center 1a Data Center 1b Data Center 2a Data Center 2b
    API Gateway SMS Payment API Gateway SMS
    2 3
    

    View Slide

12. Infrastructure > Now with Microservices
    12
    Storage
    One per instance
    Multi-Instances
    Scaling
    Upgrade
    Graceful Shutdown
    Discovery
    Session Management
    Quorum
    Data Recovery
    Stateful Service
    MariaDB
    ElasticSearch
    Kafka
    Prometheus
    Multi-Instances
    Scaling
    Upgrade
    Graceful Shutdown
    Stateless Service
    API Gateway
    Rest API
    Frontend
    Auth
    Stateless
    Stateful
    Microservice Architecture > Stateful & Stateless
    

    View Slide

13. 13
    Microservice Architecture > Kubernetes Overview
    Images
    Registry
    Charts
    Registry
    

    View Slide

14. 14
    “Turn Right” “Take me Home”
    GitOps > Declarative vs Imperative
    Imperative Declarative
    

    View Slide

15. 15
    GitOps > Infrastructure as Code
    Pull Requests
    Code
    GitOps
    Kubernetes
    Clusters
    

    View Slide

16. 16
    CI / CD > Deploy with Helm
    Your CI / CD only needs three actions:
    ● Push Images
    ● Push Charts
    ● Deploy
    

    View Slide

17. 17
    CI / CD > Deploy with Helm
    We Need:
    ● An App Image
    ● An App Chart
    ● An App Environment setup (values.yaml)
    e.g: $ helm install stable/ghost -f values.yaml
    

    View Slide

18. 18
    CI / CD > App Image Creation
    Images
    Registry
    Developer
    git checkout
    docker build
    docker push
    Code
    Repository
    

    View Slide

19. 19
    CI / CD > App Chart Creation
    Developer
    git checkout Charts
    Registry
    Chart
    Repository
    helm dep build
    helm push
    

    View Slide

20. 20
    CI / CD > Env Deployment with Helm
    git checkout Kubernetes
    Cluster
    Images
    Registry
    Charts
    Registry
    Developer
    helm repo up docker pull
    helm install
    Env
    Repository
    

    View Slide

21. 21
    CI / CD > Workflow
    Kubernetes
    Cluster
    Images
    Registry
    helm repo up
    docker pull
    helm install
    Env
    Repository
    Chart
    Repository
    Charts
    Registry
    Code
    Repository
    helm push
    docker push
    

    View Slide

22. 22
    CI / CD > Workflow
    Kubernetes
    Cluster
    Images
    Registry
    helm repo up
    docker pull
    helm install
    Env
    Repository
    Chart
    Repository
    Charts
    Registry
    Code
    Repository
    helm push
    docker push
    

    View Slide

23. 23
    Repository Patterns > Code Structure
    Project
    Repository
    Project
    Monolith Mono Repo Multi Repo
    App1
    Repository
    App2
    Repository
    ...
    App1
    App2
    Project
    Repository
    App3
    Code
    Code
    

    View Slide

24. 24
    App1
    Repository
    App3
    Repository
    App4
    Repository
    App2
    Repository
    Repository Patterns > Code Structure
    Code Code Code Code
    ALICE TEAM BOB TEAM
    

    View Slide

25. 25
    Mono Repo Multi Repo
    App1
    Repository
    App2
    Repository
    ...
    App1
    App2
    App3
    Charts
    Repository
    Chart
    Chart
    Repository Patterns > Charts Structure
    helm/charts
    e.g.
    

    View Slide

26. 26
    App1
    Repository
    Repository Patterns > Charts Structure
    Code
    Chart
    App2
    Repository
    Code
    Chart
    App3
    Repository
    Code
    Chart
    App4
    Repository
    Chart
    Code
    

    View Slide

27. 27
    Repository Patterns > Envs Structure
    Mono Repo Multi Repo
    Pre
    Environment
    Pro
    Environment
    ...
    Envs
    Repository
    dev
    pre
    pro
    pre
    pro
    : One helm release per namespace
    

    View Slide

28. 28
    Repository Patterns > Envs Structure
    App1
    Repository
    Code
    Chart
    App2
    Repository
    Code
    Chart
    App3
    Repository
    Code
    Chart
    App4
    Repository
    Code
    Chart
    Pre
    Environment
    Pro
    Environment
    Tst
    Environment
    Tst Pre Pro
    

    View Slide

29. Repository Patterns > Software Release Trains
    Tst Environment
    Tst Release
    App1
    App2
    App3
    App4
    Pre Environment
    Pre Release
    App1
    App2
    App3
    App4
    29
    Pro Environment
    Pro Release
    App1
    App2
    App3
    App4
    requirement
    s
    values.yaml values.yaml
    values.yaml
    requirements
    requirement
    s
    

    View Slide

30. Development Styles > Git Flow
    30
    develop
    feature
    branches
    release
    branches
    master
    Tag: 0.1 Tag: 1.0 Tag: 2.0
    v1 v2
    Tag: 2.1
    

    View Slide

31. Development Styles > Trunk Based
    31
    master
    feature
    branches
    release
    branches
    Tag: 1.0 Tag: 2.0
    v1 v2
    Tag: 2.1
    

    View Slide

32. 32
    CI / CD > Workflow
    Kubernetes
    Cluster
    Images
    Registry
    helm repo up
    docker pull
    helm install
    Env
    Repository
    Chart
    Repository
    Charts
    Registry
    Code
    Repository
    helm push
    docker push
    

    View Slide

33. 33
    Development Styles > Docker Image Versioning
    Each branch/tag is a potential deployment
    On each push, images are built and pushed to Registry.
    Image versioning should be immutable (if possible)
    ● Tag name is immutable
    ● Branch name is not immutable! (Immutability is important for declarative deployments)
    Two alternatives for branches deployment:
    ● Versioning should include a suffix of CI BUILD number (e.g. “master-1”)
    ● Configure Pods to redeploy always on helm upgrade (for development environments only):
    ○ Pod Annotation: helm.sh/deploy-date: "{{ .Release.Time.Seconds }}"
    ○ Pod container: imagePullPolicy: Always
    ○ Configure Registry to allow redeploys of same version
    

    View Slide

34. 34
    Development Styles > Versioning per Environment
    ● Production Deployments
    ○ by tags
    ○ continuously by “master”
    ● Staging Deployments
    ○ by “release” branches
    ○ continuously by “master”
    ○ continuously by “staging”
    ● Develop Deployments
    ○ by “develop” branch
    ○ by “feature” branches
    

    View Slide

35. 35
    Development Styles > Environments
    ● Production Environments
    ○ by cluster (AZ, DC)
    ○ by namespace (client)
    ● Staging Environments
    ○ for load testing
    ○ for security testing
    ○ for regression testing
    ● Develop Environments
    ○ by team
    ○ by developer
    ○ by feature branch
    

    View Slide

36. 36
    Development Styles > Centralized vs Decentralized
    Centralized
    Decentralized
    vs
    - Least Privilege Principle
    - Role Based Access Control
    - Self Service
    - Cluster-wide Privileges
    - Promoting silos culture
    

    View Slide

37. 37
    Demo Time!
    

    View Slide

38. 38
    DevOps Culture
    Automation
    - CI/CD
    - GitOps
    - ChatOps
    - Metrics
    - Alerts
    - Dashboards
    Feedback
    - Retrospectives
    - Transparency
    - Blameless
    post-mortems
    - Evidence
    based
    experiments
    Shared
    Responsibility
    - Objectives
    - KPIs
    - Organization
    Culture
    Autonomous
    Teams
    - Self-service
    - Build & Deploy
    Agile
    - Move fast
    - React quickly
    - Accept
    Failure
    - Fail fast
    - Fail once
    

    View Slide

39. Thank You!
    Innovating Container Delivery
    

    View Slide