1 Speed matters: Advanced CI/CD techniques to improve development velocity, quality & security Solutions Engineer, CircleCI Tadashi Nemoto

2 Self Introduction ● Tadashi Nemoto ● Solutions Engineer, CircleCI Japan & APAC ● Career ○ SET(Software Engineer in Test) at C2C marketplace app company ○ DevOps Engineer at AI startup company tadashi0713 tadashi-nemoto tadashi0713.dev

3 About this workshop Speed matters. CI/CD is a key component to improving the speed, quality and security of the overall software delivery process and being able to optimize and get the most out of your CI/CD tool will take development velocity from good to great. In this workshop I will provide a deep-dive into a variety of advanced techniques for making your pipelines run faster as well as improve quality and security. The workshop will share practical examples as well as a live demo. We will be closing the session with a Ask Me Anything Q&A.

4 Agenda ● About CircleCI ● CircleCI demo with advanced features ○ CircleCI Orb, Test splitting and parallelism, Debugging with SSH, Custom Resource Classes, Insights / Test Insights Dashboard ● Other CircleCI advanced features ○ IP Ranges, Self-hosted runners(CircleCI runner), CircleCI Server, Webhook ● Summary / Q&A

5 About CircleCI

6 Give development teams Mission the power to build and deliver software with speed and confidence

7 CircleCI Organization at a Glance Users 1M+ Employees 600+ Founded 2011 Raised $315M CircleCI Overview San Francisco, USA Tokyo, Japan Amsterdam, Netherlands Denver, USA London, UK Paris, France

8 Creation Orchestration Operations Collaborate Source Control Where CircleCI Sits in the Toolchain Deliver Build • Test • Deliver・Release Run Monitor • Operate SHIP TO PRODUCTION CODE COMMIT

9 CircleCI Strength Robust feature-set for optimization and developer efficiency Quickly and confidently validate any code changes across your projects. Data and insights for better informed decisions Monitor and improve your team’s tests. Platform performance offers fastest route to deploy Quickly and confidently validate any code changes across your projects.

10 Source project: https://github.com/strapi/strapi 10m 20m 0 30m CircleCI builds 70% faster than the competition on average. CircleCI (cloud) GitLab (cloud) GitLab (self hosted t2.medium) Buildkite (t2.medium) Travis CI AWS AppVeyor 0:04:53 0:13:25 0:17:21 0:22:37 0:28:44 2:37 0:09:19 2:37 0:21:19 2:37 Build Times for Large NodeJS Project CircleCI is focused on the productivity of development teams.

11 In software delivery, there’s no question that speed is important. Everyone wants to go fast Revenue Growth Source: McKinsey Increase in Shareholder Returns Improvement in Operating Margins Increase in Innovation 60% 20% 55% 4-5x When software teams move fast, good things happen for the business Reduce the cost of mistakes Deliver value more frequently

12 Strong features to improve development velocity ● SSH debugging ○ Securely access any job on CircleCI to debug builds and tests in real-time. ● Insights Dashboard ○ Insights allow team to use data to improve engineering productivity ● Execution environment & fleet options ○ Choose from a vast fleet of executor types: Linux, Arm, macOS, & Windows. All offer a range of CPUs, GPUs, memory, and images to customize each job. ● Test splitting ○ Shorten the feedback loop by automatically splitting your tests across parallel instances of the same job

13 CircleCI demo with advanced features

14 ● https://github.com/tadashi0713/circleci-graviton-fargate-demo ● Web application(React.js + Typescript) ○ UnitTest(Jest), E2E browser test(Playwright) ● Build Docker image → Deploy(AWS Fargate) ● CircleCI advanced features ○ CircleCI Orb ■ circleci/node, circleci/aws-ecr, circleci/aws-ecs ○ Debugging with SSH ○ Custom Resource Classes ○ Test splitting and parallelism ○ Insights / Test Insights Dashboard About demo

15 About demo

16 CircleCI Orb

17 Seamless integration of other best-in-class tools into the CircleCI platform. Integrate everywhere. Build Test Deploy Release 200+ Certified Partner Integrations 4,000 Community Integrations

18 CircleCI Orb(Language/Framework) https://circleci.com/developer/orbs/orb/circleci/node

19

20

21 Debugging with SSH

22 Debugging with SSH The best way to troubleshoot problems is to securely debug problems within the job environment - while it’s running. Without SSH access to the build environment, a developer has to try to replicate the CI/CD environment in their dev environment in order to accurately identify the issue, then attempt to resolve it using only application, stack trace, and system logs. These types of situations are a huge waste of time for developers and SRE teams.

23 Debugging with SSH - Results & Resources Resources: Angel Rivera - Debugging CI/CD Pipelines with SSH Artem Zakharchenko - Efficient CircleCI Debugging with SSH Quotes: “In my mind, CircleCI has always had the edge because of a single feature: SSH support. SSH access to jobs is really good for debugging tricky build or deploy problems...On some of the consultancy engagements I’ve worked on, this remote SSH access feels like has saved hours or even days of trial-and-error troubleshooting. Out of the box, GitHub Actions doesn’t offer the same ability to debug build problems.” - Paul Elliott, Consultant at The Scale Factory - source “Another benefit of using CircleCI is that its interactive debugging is straightforward and secure. No tokens, exposed secrets, or complex setup steps. If you don’t have Docker installed on your local machine, it is still convenient to debug CircleCI remotely. Unlike for Travis CI, it is completely secure. Unlike AppVeyor, it is easy to setup.” - John Blischak, Freelance Software Developer - source

24 Test splitting and parallelism

25 Test Splitting “Any time a developer spends waiting for tests to run is time not spent writing the next piece of code, not to mention the cost of waiting around and losing context on what they’re working on. Fast feedback is everything.” - Rob Zuber One of the easiest ways to speed up builds on CircleCI is with test splitting. Particularly, splitting tests by timing data.The CircleCI test splitting mechanism takes in a list of tests and splits those tests across the number of nodes defined by the parallelism key.

26 Customer Examples: Amio - Testing times decreased from 15mins to 9mins (40% decrease) - source Zygo - Testing times decreased from 40mins to less than 10 mins (70% decrease) - source Kogan.com - Testing times decreased from 25mins to less than 5 mins (80% decrease) - source Bolt - Testing times decreased from 15mins to less than 5 mins (33% decrease) - source Resources: Rob Zuber - Intelligent CI/CD with CircleCI: Test Splitting Ryan Pedersen - A Guide to Test Splitting Test Splitting - Results & Resources “Where CircleCI is different from other systems is their CLI tool will split your test suite for you, in a deterministic way, and distribute your test suite evenly over the number of executors you’ve declared. Most interestingly, is the test suite can be split by timing data so that each of your executors should have a fairly consistent run time. And, true to their word, we were able to get our test suite run time down to under 5 minutes.” - Josh Smeaton, Kogan.com

27 Custom Resource Classes

28 Increase developer productivity with the deepest available resource library. Build anything. Containers Linux macOS GPU Windows Self hosted runners 20+ fully-managed, instantly available resource configurations Or bring your own via runner VMs CPU x86 ARM Build Test Deploy Release https://circleci.com/product/features/resource-classes

29 Optimize resource classes with the CircleCI resources dashboard https://circleci.com/blog/optimize-resources-dashboard

30 Insights / Test Insights Dashboard

31 Insights / Test Insights Dashboard Pipeline Optimization: ● All workflow runs ● Success rate ● Duration ● Credit consumption Test Insights: ● Performance Summary ● Top 10 Most Failed Tests ● Top 10 Slowest Tests

32 https://circleci.com/docs/collect-test-data Collecting test data for Test Insights Dashboard

33 Other CircleCI advanced features

34 ● Self-hosted runners(CircleCI runner) ○ Expand your compute options to meet your most unique CI/CD needs ● CircleCI Server ○ The power of CircleCI, on-prem or in your private cloud ● IP Ranges ○ Teams are able to open up their IP-based firewalls to only CircleCI ● Webhook ○ Teams are able to receive information (referred as events) from CircleCI, as they happen Other CircleCI advanced features

35 IP Ranges

36 IP ranges: use cases = Register CircleCI IP address list https://circleci.com/docs/ip-ranges

37 How to use

38 Self-hosted runners (CircleCI runner)

39 Increase developer productivity with the deepest available resource library. Build anything. Containers Linux macOS GPU Windows Self hosted runners 20+ fully-managed, instantly available resource configurations Or bring your own via runner VMs CPU x86 ARM Build Test Deploy Release https://circleci.com/product/features/resource-classes

40

41

42 Scalable self-hosted runners on k8s(Open Preview) https://circleci.com/docs/container-runner

43 Webhook

44

45 https://docs.datadoghq.com/integrations/circleci/ https://docs.newrelic.com/docs/logs/forward-logs/circle ci-logs/ How to set up Datadog / New Relic integration

46 CircleCI Server

47 CircleCI Server Cloud Developers / users VCS (GitHub.com or Bitbucket Cloud) Database Build fleet Cache and artifacts Server Within the client’s network, CircleCI dedicated for the client is set up Developers / users VCS (GitHub.com or GitHub Enterprise Server) Database Build fleet Cache and artifacts CircleCI will set up instances and work on security and maintenance operations

48

49 CircleCI Server - Supported VCS Supported VCS VCS Cloud Server GitHub.com ✔ ✔ GitHub Enterprise Server ✗ ✔ Bitbucket Cloud ✔ ✗ Bitbucket Server ✗ ✗

50 CircleCI Server - Architecture Summary “Service cluster” (UI provision, authentication, and orchestration) Nomad cluster (Docker), VM Service, Runners (Actual job execution) VCS End users

51 Summary

52 CircleCI Strength Robust feature-set for optimization and developer efficiency Quickly and confidently validate any code changes across your projects. Data and insights for better informed decisions Monitor and improve your team’s tests. Platform performance offers fastest route to deploy Quickly and confidently validate any code changes across your projects.

53 Strong features to improve development velocity ● SSH debugging ○ Securely access any job on CircleCI to debug builds and tests in real-time. ● Insights Dashboard ○ Insights allow team to use data to improve engineering productivity ● Execution environment & fleet options ○ Choose from a vast fleet of executor types: Linux, Arm, macOS, & Windows. All offer a range of CPUs, GPUs, memory, and images to customize each job. ● Test splitting ○ Shorten the feedback loop by automatically splitting your tests across parallel instances of the same job

54 ● Self-hosted runners(CircleCI runner) ○ Expand your compute options to meet your most unique CI/CD needs ● CircleCI Server ○ The power of CircleCI, on-prem or in your private cloud ● IP Ranges ○ Teams are able to open up their IP-based firewalls to only CircleCI ● Webhook ○ Teams are able to receive information (referred as events) from CircleCI, as they happen Other CircleCI advanced features

55 Q&A

56 OpenID Connect

57 Authentication to cloud providers using static credentials

58 Authentication to cloud providers using OpenID Connect

59 Authentication to AWS using OpenID Connect

60 Authentication to AWS using OpenID Connect (CircleCI Orb) https://github.com/tadashi0713/circleci-graviton-fargate-demo https://circleci.com/developer/orbs/orb/circleci/aws-ecr