$30 off During Our Annual Pro Sale. View Details »

Speed matters: Advanced CI/CD techniques to improve development velocity, quality & security

Speed matters: Advanced CI/CD techniques to improve development velocity, quality & security

Tadashi Nemoto

October 13, 2022
Tweet

More Decks by Tadashi Nemoto

Other Decks in Programming

Transcript

  1. 1 Speed matters: Advanced CI/CD techniques to improve development velocity,

    quality & security Solutions Engineer, CircleCI Tadashi Nemoto
  2. 2 Self Introduction • Tadashi Nemoto • Solutions Engineer, CircleCI

    Japan & APAC • Career ◦ SET(Software Engineer in Test) at C2C marketplace app company ◦ DevOps Engineer at AI startup company tadashi0713 tadashi-nemoto tadashi0713.dev
  3. 3 About this workshop Speed matters. CI/CD is a key

    component to improving the speed, quality and security of the overall software delivery process and being able to optimize and get the most out of your CI/CD tool will take development velocity from good to great. In this workshop I will provide a deep-dive into a variety of advanced techniques for making your pipelines run faster as well as improve quality and security. The workshop will share practical examples as well as a live demo. We will be closing the session with a Ask Me Anything Q&A.
  4. 4 Agenda • About CircleCI • CircleCI demo with advanced

    features ◦ CircleCI Orb, Test splitting and parallelism, Debugging with SSH, Custom Resource Classes, Insights / Test Insights Dashboard • Other CircleCI advanced features ◦ IP Ranges, Self-hosted runners(CircleCI runner), CircleCI Server, Webhook • Summary / Q&A
  5. 5 About CircleCI

  6. 6 Give development teams Mission the power to build and

    deliver software with speed and confidence
  7. 7 CircleCI Organization at a Glance Users 1M+ Employees 600+

    Founded 2011 Raised $315M CircleCI Overview San Francisco, USA Tokyo, Japan Amsterdam, Netherlands Denver, USA London, UK Paris, France
  8. 8 Creation Orchestration Operations Collaborate Source Control Where CircleCI Sits

    in the Toolchain Deliver Build • Test • Deliver・Release Run Monitor • Operate SHIP TO PRODUCTION CODE COMMIT
  9. 9 CircleCI Strength Robust feature-set for optimization and developer efficiency

    Quickly and confidently validate any code changes across your projects. Data and insights for better informed decisions Monitor and improve your team’s tests. Platform performance offers fastest route to deploy Quickly and confidently validate any code changes across your projects.
  10. 10 Source project: https://github.com/strapi/strapi 10m 20m 0 30m CircleCI builds

    70% faster than the competition on average. CircleCI (cloud) GitLab (cloud) GitLab (self hosted t2.medium) Buildkite (t2.medium) Travis CI AWS AppVeyor 0:04:53 0:13:25 0:17:21 0:22:37 0:28:44 2:37 0:09:19 2:37 0:21:19 2:37 Build Times for Large NodeJS Project CircleCI is focused on the productivity of development teams.
  11. 11 In software delivery, there’s no question that speed is

    important. Everyone wants to go fast Revenue Growth Source: McKinsey Increase in Shareholder Returns Improvement in Operating Margins Increase in Innovation 60% 20% 55% 4-5x When software teams move fast, good things happen for the business Reduce the cost of mistakes Deliver value more frequently
  12. 12 Strong features to improve development velocity • SSH debugging

    ◦ Securely access any job on CircleCI to debug builds and tests in real-time. • Insights Dashboard ◦ Insights allow team to use data to improve engineering productivity • Execution environment & fleet options ◦ Choose from a vast fleet of executor types: Linux, Arm, macOS, & Windows. All offer a range of CPUs, GPUs, memory, and images to customize each job. • Test splitting ◦ Shorten the feedback loop by automatically splitting your tests across parallel instances of the same job
  13. 13 CircleCI demo with advanced features

  14. 14 • https://github.com/tadashi0713/circleci-graviton-fargate-demo • Web application(React.js + Typescript) ◦ UnitTest(Jest),

    E2E browser test(Playwright) • Build Docker image → Deploy(AWS Fargate) • CircleCI advanced features ◦ CircleCI Orb ▪ circleci/node, circleci/aws-ecr, circleci/aws-ecs ◦ Debugging with SSH ◦ Custom Resource Classes ◦ Test splitting and parallelism ◦ Insights / Test Insights Dashboard About demo
  15. 15 About demo

  16. 16 CircleCI Orb

  17. 17 Seamless integration of other best-in-class tools into the CircleCI

    platform. Integrate everywhere. Build Test Deploy Release 200+ Certified Partner Integrations 4,000 Community Integrations
  18. 18 CircleCI Orb(Language/Framework) https://circleci.com/developer/orbs/orb/circleci/node

  19. 19

  20. 20

  21. 21 Debugging with SSH

  22. 22 Debugging with SSH The best way to troubleshoot problems

    is to securely debug problems within the job environment - while it’s running. Without SSH access to the build environment, a developer has to try to replicate the CI/CD environment in their dev environment in order to accurately identify the issue, then attempt to resolve it using only application, stack trace, and system logs. These types of situations are a huge waste of time for developers and SRE teams.
  23. 23 Debugging with SSH - Results & Resources Resources: Angel

    Rivera - Debugging CI/CD Pipelines with SSH Artem Zakharchenko - Efficient CircleCI Debugging with SSH Quotes: “In my mind, CircleCI has always had the edge because of a single feature: SSH support. SSH access to jobs is really good for debugging tricky build or deploy problems...On some of the consultancy engagements I’ve worked on, this remote SSH access feels like has saved hours or even days of trial-and-error troubleshooting. Out of the box, GitHub Actions doesn’t offer the same ability to debug build problems.” - Paul Elliott, Consultant at The Scale Factory - source “Another benefit of using CircleCI is that its interactive debugging is straightforward and secure. No tokens, exposed secrets, or complex setup steps. If you don’t have Docker installed on your local machine, it is still convenient to debug CircleCI remotely. Unlike for Travis CI, it is completely secure. Unlike AppVeyor, it is easy to setup.” - John Blischak, Freelance Software Developer - source
  24. 24 Test splitting and parallelism

  25. 25 Test Splitting “Any time a developer spends waiting for

    tests to run is time not spent writing the next piece of code, not to mention the cost of waiting around and losing context on what they’re working on. Fast feedback is everything.” - Rob Zuber One of the easiest ways to speed up builds on CircleCI is with test splitting. Particularly, splitting tests by timing data.The CircleCI test splitting mechanism takes in a list of tests and splits those tests across the number of nodes defined by the parallelism key.
  26. 26 Customer Examples: Amio - Testing times decreased from 15mins

    to 9mins (40% decrease) - source Zygo - Testing times decreased from 40mins to less than 10 mins (70% decrease) - source Kogan.com - Testing times decreased from 25mins to less than 5 mins (80% decrease) - source Bolt - Testing times decreased from 15mins to less than 5 mins (33% decrease) - source Resources: Rob Zuber - Intelligent CI/CD with CircleCI: Test Splitting Ryan Pedersen - A Guide to Test Splitting Test Splitting - Results & Resources “Where CircleCI is different from other systems is their CLI tool will split your test suite for you, in a deterministic way, and distribute your test suite evenly over the number of executors you’ve declared. Most interestingly, is the test suite can be split by timing data so that each of your executors should have a fairly consistent run time. And, true to their word, we were able to get our test suite run time down to under 5 minutes.” - Josh Smeaton, Kogan.com
  27. 27 Custom Resource Classes

  28. 28 Increase developer productivity with the deepest available resource library.

    Build anything. Containers Linux macOS GPU Windows Self hosted runners 20+ fully-managed, instantly available resource configurations Or bring your own via runner VMs CPU x86 ARM Build Test Deploy Release https://circleci.com/product/features/resource-classes
  29. 29 Optimize resource classes with the CircleCI resources dashboard https://circleci.com/blog/optimize-resources-dashboard

  30. 30 Insights / Test Insights Dashboard

  31. 31 Insights / Test Insights Dashboard Pipeline Optimization: • All

    workflow runs • Success rate • Duration • Credit consumption Test Insights: • Performance Summary • Top 10 Most Failed Tests • Top 10 Slowest Tests
  32. 32 https://circleci.com/docs/collect-test-data Collecting test data for Test Insights Dashboard

  33. 33 Other CircleCI advanced features

  34. 34 • Self-hosted runners(CircleCI runner) ◦ Expand your compute options

    to meet your most unique CI/CD needs • CircleCI Server ◦ The power of CircleCI, on-prem or in your private cloud • IP Ranges ◦ Teams are able to open up their IP-based firewalls to only CircleCI • Webhook ◦ Teams are able to receive information (referred as events) from CircleCI, as they happen Other CircleCI advanced features
  35. 35 IP Ranges

  36. 36 IP ranges: use cases = Register CircleCI IP address

    list https://circleci.com/docs/ip-ranges
  37. 37 How to use

  38. 38 Self-hosted runners (CircleCI runner)

  39. 39 Increase developer productivity with the deepest available resource library.

    Build anything. Containers Linux macOS GPU Windows Self hosted runners 20+ fully-managed, instantly available resource configurations Or bring your own via runner VMs CPU x86 ARM Build Test Deploy Release https://circleci.com/product/features/resource-classes
  40. 40

  41. 41

  42. 42 Scalable self-hosted runners on k8s(Open Preview) https://circleci.com/docs/container-runner

  43. 43 Webhook

  44. 44

  45. 45 https://docs.datadoghq.com/integrations/circleci/ https://docs.newrelic.com/docs/logs/forward-logs/circle ci-logs/ How to set up Datadog /

    New Relic integration
  46. 46 CircleCI Server

  47. 47 CircleCI Server Cloud Developers / users VCS (GitHub.com or

    Bitbucket Cloud) Database Build fleet Cache and artifacts Server Within the client’s network, CircleCI dedicated for the client is set up Developers / users VCS (GitHub.com or GitHub Enterprise Server) Database Build fleet Cache and artifacts CircleCI will set up instances and work on security and maintenance operations
  48. 48

  49. 49 CircleCI Server - Supported VCS Supported VCS VCS Cloud

    Server GitHub.com ✔ ✔ GitHub Enterprise Server ✗ ✔ Bitbucket Cloud ✔ ✗ Bitbucket Server ✗ ✗
  50. 50 CircleCI Server - Architecture Summary “Service cluster” (UI provision,

    authentication, and orchestration) Nomad cluster (Docker), VM Service, Runners (Actual job execution) VCS End users
  51. 51 Summary

  52. 52 CircleCI Strength Robust feature-set for optimization and developer efficiency

    Quickly and confidently validate any code changes across your projects. Data and insights for better informed decisions Monitor and improve your team’s tests. Platform performance offers fastest route to deploy Quickly and confidently validate any code changes across your projects.
  53. 53 Strong features to improve development velocity • SSH debugging

    ◦ Securely access any job on CircleCI to debug builds and tests in real-time. • Insights Dashboard ◦ Insights allow team to use data to improve engineering productivity • Execution environment & fleet options ◦ Choose from a vast fleet of executor types: Linux, Arm, macOS, & Windows. All offer a range of CPUs, GPUs, memory, and images to customize each job. • Test splitting ◦ Shorten the feedback loop by automatically splitting your tests across parallel instances of the same job
  54. 54 • Self-hosted runners(CircleCI runner) ◦ Expand your compute options

    to meet your most unique CI/CD needs • CircleCI Server ◦ The power of CircleCI, on-prem or in your private cloud • IP Ranges ◦ Teams are able to open up their IP-based firewalls to only CircleCI • Webhook ◦ Teams are able to receive information (referred as events) from CircleCI, as they happen Other CircleCI advanced features
  55. 55 Q&A

  56. 56 OpenID Connect

  57. 57 Authentication to cloud providers using static credentials

  58. 58 Authentication to cloud providers using OpenID Connect

  59. 59 Authentication to AWS using OpenID Connect

  60. 60 Authentication to AWS using OpenID Connect (CircleCI Orb) https://github.com/tadashi0713/circleci-graviton-fargate-demo

    https://circleci.com/developer/orbs/orb/circleci/aws-ecr