Best Practice CI/CD Pipeline for Deploying Container Apps to AWS

Slide 1

Slide 1 text

1 Best Practice CI/CD Pipeline for Deploying Container Apps to AWS 2024/03/20 Tadashi Nemoto Senior Solutions Engineer

Slide 2

Slide 2 text

2 Self-introduction ● Tadashi Nemoto ● Senior Solutions Engineer, Japan & APAC @CircleCI ● Career ○ SET(Software Engineer in Test) at C2C marketplace app company ○ DevOps Engineer at AI startup company tadashi0713 tadashi-nemoto tadashi0713.dev

Slide 3

Slide 3 text

3 Creation Orchestration Operations Collaborate Source Control About CircleCI Deliver Build • Test • Deliver・Release Run Monitor • Operate SHIP TO PRODUCTION CODE COMMIT

Slide 4

Slide 4 text

4 About this session AWS provides strong products and support to deploy container apps, such as AWS ECS, and AWS EKS(Kubernetes) However, there are several ways and strategies to deploy/release to these platforms, and it's hard to decide which way is right for your organization. In this session, I will share some best practices of the CI/CD pipeline for deploying container apps to AWS to improve your company's developer productivity, governance, and security.

Slide 5

Slide 5 text

5 Agenda Basic understanding of CI/CD Pipeline for Deploying Container Apps to AWS Best practice CI/CD pipeline for Startups Summary 1 2 3 4 Best practice CI/CD pipeline for Enterprises

Slide 6

Slide 6 text

6 Basic understanding of CI/CD Pipeline for Deploying Container Apps to AWS

Slide 7

Slide 7 text

https://www.qovery.com/blog/deploying-containers-on-aws-elastic-beanstalk-vs-ecs-vs-eks/ AWS ECS vs AWS EKS

Slide 8

Slide 8 text

8 AWS EC2 vs AWS Fargate(Serverless) https://aws.amazon.com/fargate/

Slide 9

Slide 9 text

9 Basic CI/CD Architecture for AWS ECS / AWS EKS

Slide 10

Slide 10 text

10 Progressive Delivery https://argo-rollouts.readthedocs.io/en/stable/concepts/ Canary Blue-Green

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Image credit - AngelList

Slide 13

Slide 13 text

13 Best Practice CI/CD Pipeline for Startups

Slide 14

Slide 14 text

14 Situations in Startups ● ~30 developers ● No dedicated DevOps / SRE Engineers ● Simple application / infrastructure architecture ○ Frontend + Backend + DB ○ Only AWS ● Speed(Lead time, Deployment frequency) is important

Slide 15

Slide 15 text

● Kubernetes + multi-cloud!! ● Microservices!! ● Progressive Delivery!! ● Self-hosted runners on k8s!! ● Platform Engineering!! ● Custom Templates!! Complexity × Maintenance = Block Speed…

Slide 16

Slide 16 text

16 Best Practice CI/CD Pipeline for Startups 1. Choose the low maintenance platform ○ AWS ECS on AWS Fargate ○ Single & Cloud-hosted CI/CD tools 2. Utilize AWS integrations to avoid reinventing the wheel ○ CircleCI integrations(Orb) for AWS 3. Simple CI/CD pipeline is best ○ Progressive Delivery is unnecessary ○ Focus on app quality(Unit Test)

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

18 CircleCI Orb for AWS https://circleci.com/integrations/aws/

Slide 19

Slide 19 text

Slide 20

Slide 20 text

20 Focus on app quality(Unit Test) Speed up test execution time (Test Splitting) Shorten the feedback loop by automatically splitting your tests across parallel instances of the same job Continuously measure test results and coverage CircleCI Test Insight Dashboard continuously measure test results with flaky test detection CircleCI(Orb) can integrate 3rd party tools(SonarCloud, Codecov) to continuously measure test coverage

Slide 21

Slide 21 text

21 Best Practice CI/CD Pipeline for Enterprises

Slide 22

Slide 22 text

22 Situations in Enterprises ● 100~ developers ● Dedicated DevOps / SRE Teams ● Complex applications / infrastructure architecture ○ Multi-Cloud, On-prem ○ Several services(Microservices) ● Reliability(Change failure rate, Mean time to recovery) and Governance are important

Slide 23

Slide 23 text

23 Each team is using different architecture & tools… Poor governance, visibility, productivity… Team A Team B Team C

Slide 24

Slide 24 text

24 Best Practice CI/CD Pipeline for Enterprises 1. Choose tools as organization’s application platform(Platform Engineering) ○ AWS EKS(Kubernetes) ○ Hybrid / Multi-support CI/CD 2. Improve Visibility and Consistency ○ Insights Dashboard, CircleCI Releases 3. Consider Progressive Delivery to reduce MTTR & improve reliability

Slide 25

Slide 25 text

Team A Platform Teams Self-serve ● Container Platform(Amazon EKS) ● Version Control System(Git) ● CI/CD ○ Pipeline data(Usage, Success rate) ○ Custom Templates ○ Release management ○ Governance(Policy, Audit log) ● Monitoring, Secret manager, etc. Better governance, visibility, productivity Team A Team A

Slide 26

Slide 26 text

26 Insights / Test Insights Dashboard Pipeline Optimization: ● All workﬂow runs ● Success rate ● Duration ● Credit consumption Test Insights: ● Performance Summary ● Top 10 Most Failed Tests ● Top 10 Slowest Tests

Slide 27

Slide 27 text

28 CircleCI Releases supporting Progressive Delivery(Argo Rollouts)

Slide 28

Slide 28 text

30 Summary

Slide 29

Slide 29 text

31 Summary ● The strategy and best practices for CI/CD vary depending on the size of the organization. ● Startups(Speed) ○ Low maintenance platform ○ Utilize AWS integrations ○ Simple CI/CD is best(Focus on app quality) ● Enterprise(Reliability, Governance) ○ Choose tools as organization’s platform ○ Improve Visibility and Consistency ○ Consider Progressive Delivery

Slide 30

Slide 30 text

32 Thank you!! tadashi@circleci.com