10x deployment frequency using GitLab Flow and GitHub Actions on AWS ECS

Slide 1

Slide 1 text

10x deployment frequency using GitLab Flow and GitHub Actions on AWS ECS Tadashi Nemoto

Slide 2

Slide 2 text

Introduction ● Tadashi Nemoto ● Exawizards, Inc. ● Platform Engineer (DevOps Engineer) ○ Introduce and improve CI / CD pipelines ○ Maintain production / develop environments ○ Introduce Test Automation

Slide 3

Slide 3 text

Have you introduced CI / CD ?

Slide 4

Slide 4 text

Are these CI / CD pipelines optimized for current product / organization?

Slide 5

Slide 5 text

State of DevOps Report 2019

Slide 6

Slide 6 text

State of DevOps Report 2018

Slide 7

Slide 7 text

State of DevOps Report 2018

Slide 8

Slide 8 text

Startup companies need speed to market

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

Deployment Frequency(27 deploys / 2 weeks) 10x Deployment Frequency

Slide 11

Slide 11 text

Outlines ● Previous CI /CD pipelines and deployment flows ● What I improved ○ Jenkins → GitHub Actions on AWS ECS ○ Git Flow → GitLab Flow ● Effect of improvements ● Challenges

Slide 12

Slide 12 text

Previous CI / CD pipelines and deployment flows

Slide 13

Slide 13 text

Previous CI / CD・Deployment flows ● Hashicorp Nomad on AWS ○ develop, staging, production environments ○ Difficult to create multiple development environments ● Git Flow as branching strategy ○ Usages are slightly different depending on teams ● Jenkins on AWS ○ Some deployments to production depend on us

Slide 14

Slide 14 text

Enable small, autonomous deployments

Slide 15

Slide 15 text

Increase deployment frequency

Slide 16

Slide 16 text

Improvement ① Jenkins → GitHub Actions on AWS ECS

Slide 17

Slide 17 text

Jenkins ● Maintenance cost ○ Update versions, plugins ○ Add machines, scale ○ Authority management, Security, etc. ● Need dedicated members or teams ● Not suitable for autonomous deployments

Slide 18

Slide 18 text

SaaS CI / CD tools

Slide 19

Slide 19 text

Limitation of deployment

Slide 20

Slide 20 text

GitHub Actions self-hosted runners

Slide 21

Slide 21 text

GitHub Actions self-hosted runners ● GitHub Actions provides self-hosted runners ● No additional charge ● Compatible both self-hosted and GitHub-hosted runners ○ Deploy → self-hosted runners ○ Test → GitHub-hosted runners ● No need to maintain the management part of a CI / CD workflow (like a Jenkins master instance)

Slide 22

Slide 22 text

GitHub Actions self-hosted runners on AWS ECS

Slide 23

Slide 23 text

https://techblog.exawizards.com/entry/2020/11/04/100000

Slide 24

Slide 24 text

Improvement ② Git Flow → GitLab Flow

Slide 25

Slide 25 text

Git Flow

Slide 26

Slide 26 text

Git Flow ● Works well for projects with fixed release timing ○ Mobile apps(1 release / 1~2 weeks) ● Less beneficial for projects API / Frontend projects ● More complex branch management, and not suitable for deploying to production frequently ○ Creating release branches, Hotfix branches, Tags

Slide 27

Slide 27 text

GitHub Flow

Slide 28

Slide 28 text

GitHub Flow Prod env ︖env ● Simpler branch management ○ master / feature branch ● Enable high deployment frequency ● Problems with verification environment ○ master branch = prod environment ○ staging environment︖ ○ development environment︖ ︖env

Slide 29

Slide 29 text

I want high deployment frequency while utilizing current environments (develop, staging, production)

Slide 30

Slide 30 text

GitLab Flow ● feature branch → merge to master branch ○ Same as GitHub Flow ● GitLab Flow allows to have branch for a release ○ master branch → staging env ○ production branch → production env ○ Merge to production branch for production release Staging env Prod env

Slide 31

Slide 31 text

Deploy to develop environment (workflow_dispatch)

Slide 32

Slide 32 text

Automatically generate release pull requests Staging env Prod env

Slide 33

Slide 33 text

https://techblog.exawizards.com/entry/2021/01/28/113812

Slide 34

Slide 34 text

Effect of improvements

Slide 35

Slide 35 text

Deployment Frequency(4 deploys / 1 month)

Slide 36

Slide 36 text

Deployment Frequency(27 deploys / 2 weeks)

Slide 37

Slide 37 text

Deployment Frequency(27 deploys / 2 weeks) 10x Deployment Frequency

Slide 38

Slide 38 text

State of DevOps Report 2019

Slide 39

Slide 39 text

Small, autonomous deployments

Slide 40

Slide 40 text

Challenges

Slide 41

Slide 41 text

Continuous monitoring / improvement

Slide 42

Slide 42 text

PR based environments / drop self-hosted runners Staging env PR1 env PR2 env

Slide 43

Slide 43 text

Continuous Testing / Inspection

Slide 44

Slide 44 text

Conclusion

Slide 45

Slide 45 text

Message(Key takeaways) ● Important to improve deployment frequency, especially startups ● I achieved 10x deployment frequency by 2 improvements ○ Jenkins → GitHub Actions on AWS ECS ○ Git Flow → GitLab Flow ● Continuous monitoring / improvement