Best Practice CI/CD Pipeline for Deploying Container Apps to AWS

AWS User Group Singapore: AWS Meetup March 2024 (AWS Office)
https://www.meetup.com/aws-sg/events/299504689/

[Description]
AWS provides strong products and support to deploy container apps, such as AWS ECS, and AWS EKS(Kubernetes)

However, there are several ways and strategies to deploy/release to these platforms, and it's hard to decide which way is right for your organization.

In this session, I will share some best practices of the CI/CD pipeline for deploying container apps to AWS to improve your company's developer productivity, governance, and security.

Transcript

1. 1 Best Practice CI/CD Pipeline for Deploying Container Apps to AWS 2024/03/20 Tadashi Nemoto Senior Solutions Engineer
2. 2 Self-introduction • Tadashi Nemoto • Senior Solutions Engineer, Japan & APAC @CircleCI • Career ◦ SET(Software Engineer in Test) at C2C marketplace app company ◦ DevOps Engineer at AI startup company tadashi0713 tadashi-nemoto tadashi0713.dev
3. 3 Creation Orchestration Operations Collaborate Source Control About CircleCI Deliver Build • Test • Deliver・Release Run Monitor • Operate SHIP TO PRODUCTION CODE COMMIT
4. 4 About this session AWS provides strong products and support to deploy container apps, such as AWS ECS, and AWS EKS(Kubernetes) However, there are several ways and strategies to deploy/release to these platforms, and it's hard to decide which way is right for your organization. In this session, I will share some best practices of the CI/CD pipeline for deploying container apps to AWS to improve your company's developer productivity, governance, and security.
5. 5 Agenda Basic understanding of CI/CD Pipeline for Deploying Container Apps to AWS Best practice CI/CD pipeline for Startups Summary 1 2 3 4 Best practice CI/CD pipeline for Enterprises
6. 6 Basic understanding of CI/CD Pipeline for Deploying Container Apps to AWS
7. https://www.qovery.com/blog/deploying-containers-on-aws-elastic-beanstalk-vs-ecs-vs-eks/ AWS ECS vs AWS EKS
8. 8 AWS EC2 vs AWS Fargate(Serverless) https://aws.amazon.com/fargate/
9. 9 Basic CI/CD Architecture for AWS ECS / AWS EKS
10. 10 Progressive Delivery https://argo-rollouts.readthedocs.io/en/stable/concepts/ Canary Blue-Green
12. Image credit - AngelList
13. 13 Best Practice CI/CD Pipeline for Startups
14. 14 Situations in Startups • ~30 developers • No dedicated DevOps / SRE Engineers • Simple application / infrastructure architecture ◦ Frontend + Backend + DB ◦ Only AWS • Speed(Lead time, Deployment frequency) is important
15. • Kubernetes + multi-cloud!! • Microservices!! • Progressive Delivery!! • Self-hosted runners on k8s!! • Platform Engineering!! • Custom Templates!! Complexity × Maintenance = Block Speed…
16. 16 Best Practice CI/CD Pipeline for Startups 1. Choose the low maintenance platform ◦ AWS ECS on AWS Fargate ◦ Single & Cloud-hosted CI/CD tools 2. Utilize AWS integrations to avoid reinventing the wheel ◦ CircleCI integrations(Orb) for AWS 3. Simple CI/CD pipeline is best ◦ Progressive Delivery is unnecessary ◦ Focus on app quality(Unit Test)
18. 18 CircleCI Orb for AWS https://circleci.com/integrations/aws/
20. 20 Focus on app quality(Unit Test) Speed up test execution time (Test Splitting) Shorten the feedback loop by automatically splitting your tests across parallel instances of the same job Continuously measure test results and coverage CircleCI Test Insight Dashboard continuously measure test results with flaky test detection CircleCI(Orb) can integrate 3rd party tools(SonarCloud, Codecov) to continuously measure test coverage
21. 21 Best Practice CI/CD Pipeline for Enterprises
22. 22 Situations in Enterprises • 100~ developers • Dedicated DevOps / SRE Teams • Complex applications / infrastructure architecture ◦ Multi-Cloud, On-prem ◦ Several services(Microservices) • Reliability(Change failure rate, Mean time to recovery) and Governance are important
23. 23 Each team is using different architecture & tools… Poor governance, visibility, productivity… Team A Team B Team C
24. 24 Best Practice CI/CD Pipeline for Enterprises 1. Choose tools as organization’s application platform(Platform Engineering) ◦ AWS EKS(Kubernetes) ◦ Hybrid / Multi-support CI/CD 2. Improve Visibility and Consistency ◦ Insights Dashboard, CircleCI Releases 3. Consider Progressive Delivery to reduce MTTR & improve reliability
25. Team A Platform Teams Self-serve • Container Platform(Amazon EKS) • Version Control System(Git) • CI/CD ◦ Pipeline data(Usage, Success rate) ◦ Custom Templates ◦ Release management ◦ Governance(Policy, Audit log) • Monitoring, Secret manager, etc. Better governance, visibility, productivity Team A Team A
26. 26 Insights / Test Insights Dashboard Pipeline Optimization: • All workflow runs • Success rate • Duration • Credit consumption Test Insights: • Performance Summary • Top 10 Most Failed Tests • Top 10 Slowest Tests
27. 28 CircleCI Releases supporting Progressive Delivery(Argo Rollouts)
28. 30 Summary
29. 31 Summary • The strategy and best practices for CI/CD vary depending on the size of the organization. • Startups(Speed) ◦ Low maintenance platform ◦ Utilize AWS integrations ◦ Simple CI/CD is best(Focus on app quality) • Enterprise(Reliability, Governance) ◦ Choose tools as organization’s platform ◦ Improve Visibility and Consistency ◦ Consider Progressive Delivery
30. 32 Thank you!! tadashi@circleci.com