Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Speed matters: Advanced CI/CD techniques to improve development velocity, quality & security

Speed matters: Advanced CI/CD techniques to improve development velocity, quality & security

Tadashi Nemoto

October 13, 2022
Tweet

More Decks by Tadashi Nemoto

Other Decks in Programming

Transcript

  1. 1
    Speed matters:
    Advanced CI/CD techniques to
    improve development velocity,
    quality & security
    Solutions Engineer, CircleCI
    Tadashi Nemoto

    View full-size slide

  2. 2
    Self Introduction
    ● Tadashi Nemoto
    ● Solutions Engineer, CircleCI Japan & APAC
    ● Career
    ○ SET(Software Engineer in Test) at C2C
    marketplace app company
    ○ DevOps Engineer at AI startup company
    tadashi0713
    tadashi-nemoto
    tadashi0713.dev

    View full-size slide

  3. 3
    About this workshop
    Speed matters. CI/CD is a key component to improving the speed, quality and
    security of the overall software delivery process and being able to optimize and
    get the most out of your CI/CD tool will take development velocity from good to
    great.
    In this workshop I will provide a deep-dive into a variety of advanced techniques
    for making your pipelines run faster as well as improve quality and security.
    The workshop will share practical examples as well as a live demo.
    We will be closing the session with a Ask Me Anything Q&A.

    View full-size slide

  4. 4
    Agenda
    ● About CircleCI
    ● CircleCI demo with advanced features
    ○ CircleCI Orb, Test splitting and parallelism, Debugging with SSH,
    Custom Resource Classes, Insights / Test Insights Dashboard
    ● Other CircleCI advanced features
    ○ IP Ranges, Self-hosted runners(CircleCI runner), CircleCI Server,
    Webhook
    ● Summary / Q&A

    View full-size slide

  5. 5
    About CircleCI

    View full-size slide

  6. 6
    Give development teams
    Mission
    the power to build and deliver software
    with speed and confidence

    View full-size slide

  7. 7
    CircleCI Organization at a Glance
    Users
    1M+
    Employees
    600+
    Founded
    2011
    Raised
    $315M
    CircleCI Overview
    San Francisco, USA Tokyo, Japan
    Amsterdam, Netherlands
    Denver, USA London, UK
    Paris, France

    View full-size slide

  8. 8
    Creation Orchestration Operations
    Collaborate
    Source Control
    Where CircleCI Sits in the Toolchain
    Deliver
    Build • Test • Deliver・Release
    Run
    Monitor • Operate
    SHIP TO PRODUCTION
    CODE COMMIT

    View full-size slide

  9. 9
    CircleCI Strength
    Robust feature-set for
    optimization and
    developer efficiency
    Quickly and confidently
    validate any code changes
    across your projects.
    Data and insights for
    better informed decisions
    Monitor and improve your
    team’s tests.
    Platform performance
    offers fastest route to
    deploy
    Quickly and confidently
    validate any code changes
    across your projects.

    View full-size slide

  10. 10
    Source project: https://github.com/strapi/strapi
    10m 20m
    0 30m
    CircleCI builds 70% faster than the competition on average.
    CircleCI (cloud)
    GitLab (cloud)
    GitLab (self hosted
    t2.medium)
    Buildkite (t2.medium)
    Travis CI
    AWS
    AppVeyor
    0:04:53
    0:13:25
    0:17:21
    0:22:37
    0:28:44
    2:37
    0:09:19
    2:37
    0:21:19
    2:37
    Build Times for Large NodeJS Project
    CircleCI is focused on the productivity of development teams.

    View full-size slide

  11. 11
    In software delivery, there’s no
    question that speed is important.
    Everyone wants to go fast
    Revenue Growth
    Source: McKinsey
    Increase in Shareholder Returns
    Improvement in Operating Margins
    Increase in Innovation
    60%
    20%
    55%
    4-5x
    When software teams move fast, good
    things happen for the business
    Reduce the
    cost of mistakes
    Deliver value
    more frequently

    View full-size slide

  12. 12
    Strong features to improve development velocity
    ● SSH debugging
    ○ Securely access any job on CircleCI to debug builds and tests in
    real-time.
    ● Insights Dashboard
    ○ Insights allow team to use data to improve engineering
    productivity
    ● Execution environment & fleet options
    ○ Choose from a vast fleet of executor types:
    Linux, Arm, macOS, & Windows.
    All offer a range of CPUs, GPUs, memory, and
    images to customize each job.
    ● Test splitting
    ○ Shorten the feedback loop by automatically
    splitting your tests across parallel instances of
    the same job

    View full-size slide

  13. 13
    CircleCI demo with advanced
    features

    View full-size slide

  14. 14
    ● https://github.com/tadashi0713/circleci-graviton-fargate-demo
    ● Web application(React.js + Typescript)
    ○ UnitTest(Jest), E2E browser test(Playwright)
    ● Build Docker image → Deploy(AWS Fargate)
    ● CircleCI advanced features
    ○ CircleCI Orb
    ■ circleci/node, circleci/aws-ecr, circleci/aws-ecs
    ○ Debugging with SSH
    ○ Custom Resource Classes
    ○ Test splitting and parallelism
    ○ Insights / Test Insights Dashboard
    About demo

    View full-size slide

  15. 15
    About demo

    View full-size slide

  16. 16
    CircleCI Orb

    View full-size slide

  17. 17
    Seamless integration of other best-in-class tools into the CircleCI platform.
    Integrate everywhere.
    Build Test Deploy Release
    200+ Certified Partner Integrations 4,000 Community Integrations

    View full-size slide

  18. 18
    CircleCI Orb(Language/Framework)
    https://circleci.com/developer/orbs/orb/circleci/node

    View full-size slide

  19. 21
    Debugging with SSH

    View full-size slide

  20. 22
    Debugging with SSH
    The best way to troubleshoot problems is to securely debug
    problems within the job environment - while it’s running.
    Without SSH access to the build environment, a developer
    has to try to replicate the CI/CD environment in their dev
    environment in order to accurately identify the issue, then
    attempt to resolve it using only application, stack trace, and
    system logs.
    These types of situations are a huge waste of time for
    developers and SRE teams.

    View full-size slide

  21. 23
    Debugging with SSH - Results & Resources
    Resources:
    Angel Rivera - Debugging CI/CD Pipelines with SSH
    Artem Zakharchenko - Efficient CircleCI Debugging with SSH
    Quotes:
    “In my mind, CircleCI has always had the edge because of a single feature: SSH support. SSH access to jobs is really
    good for debugging tricky build or deploy problems...On some of the consultancy engagements I’ve worked on, this
    remote SSH access feels like has saved hours or even days of trial-and-error troubleshooting. Out of the box, GitHub
    Actions doesn’t offer the same ability to debug build problems.”
    - Paul Elliott, Consultant at The Scale Factory - source
    “Another benefit of using CircleCI is that its interactive debugging is straightforward and secure. No tokens, exposed
    secrets, or complex setup steps. If you don’t have Docker installed on your local machine, it is still convenient to debug
    CircleCI remotely. Unlike for Travis CI, it is completely secure. Unlike AppVeyor, it is easy to setup.”
    - John Blischak, Freelance Software Developer - source

    View full-size slide

  22. 24
    Test splitting and parallelism

    View full-size slide

  23. 25
    Test Splitting
    “Any time a developer spends waiting
    for tests to run is time not spent writing
    the next piece of code, not to mention
    the cost of waiting around and losing
    context on what they’re working on.
    Fast feedback is everything.”
    - Rob Zuber
    One of the easiest ways to speed up builds on CircleCI is with
    test splitting. Particularly, splitting tests by timing data.The
    CircleCI test splitting mechanism takes in a list of tests and
    splits those tests across the number of nodes defined by the
    parallelism key.

    View full-size slide

  24. 26
    Customer Examples:
    Amio - Testing times decreased from 15mins to 9mins (40%
    decrease) - source
    Zygo - Testing times decreased from 40mins to less than 10
    mins (70% decrease) - source
    Kogan.com - Testing times decreased from 25mins to less
    than 5 mins (80% decrease) - source
    Bolt - Testing times decreased from 15mins to less than 5
    mins (33% decrease) - source
    Resources:
    Rob Zuber - Intelligent CI/CD with CircleCI: Test Splitting
    Ryan Pedersen - A Guide to Test Splitting
    Test Splitting - Results &
    Resources
    “Where CircleCI is different from other
    systems is their CLI tool will split your test
    suite for you, in a deterministic way, and
    distribute your test suite evenly over the
    number of executors you’ve declared.
    Most interestingly, is the test suite can be
    split by timing data so that each of your
    executors should have a fairly consistent
    run time. And, true to their word, we were
    able to get our test suite run time down to
    under 5 minutes.”
    - Josh Smeaton, Kogan.com

    View full-size slide

  25. 27
    Custom Resource Classes

    View full-size slide

  26. 28
    Increase developer productivity with the deepest available resource library.
    Build anything.
    Containers
    Linux macOS
    GPU
    Windows
    Self hosted
    runners
    20+ fully-managed, instantly available resource configurations Or bring your own via runner
    VMs
    CPU
    x86 ARM
    Build Test Deploy Release
    https://circleci.com/product/features/resource-classes

    View full-size slide

  27. 29
    Optimize resource classes with the CircleCI resources
    dashboard
    https://circleci.com/blog/optimize-resources-dashboard

    View full-size slide

  28. 30
    Insights / Test Insights
    Dashboard

    View full-size slide

  29. 31
    Insights / Test Insights Dashboard
    Pipeline Optimization:
    ● All workflow runs
    ● Success rate
    ● Duration
    ● Credit consumption
    Test Insights:
    ● Performance Summary
    ● Top 10 Most Failed Tests
    ● Top 10 Slowest Tests

    View full-size slide

  30. 32
    https://circleci.com/docs/collect-test-data
    Collecting test data for Test Insights Dashboard

    View full-size slide

  31. 33
    Other CircleCI advanced
    features

    View full-size slide

  32. 34
    ● Self-hosted runners(CircleCI runner)
    ○ Expand your compute options to meet your most unique CI/CD
    needs
    ● CircleCI Server
    ○ The power of CircleCI, on-prem or in your private cloud
    ● IP Ranges
    ○ Teams are able to open up their IP-based
    firewalls to only CircleCI
    ● Webhook
    ○ Teams are able to receive information (referred
    as events) from CircleCI, as they happen
    Other CircleCI advanced features

    View full-size slide

  33. 36
    IP ranges: use cases
    = Register CircleCI IP address list
    https://circleci.com/docs/ip-ranges

    View full-size slide

  34. 37
    How to use

    View full-size slide

  35. 38
    Self-hosted runners
    (CircleCI runner)

    View full-size slide

  36. 39
    Increase developer productivity with the deepest available resource library.
    Build anything.
    Containers
    Linux macOS
    GPU
    Windows
    Self hosted
    runners
    20+ fully-managed, instantly available resource configurations Or bring your own via runner
    VMs
    CPU
    x86 ARM
    Build Test Deploy Release
    https://circleci.com/product/features/resource-classes

    View full-size slide

  37. 42
    Scalable self-hosted runners on k8s(Open Preview)
    https://circleci.com/docs/container-runner

    View full-size slide

  38. 45
    https://docs.datadoghq.com/integrations/circleci/ https://docs.newrelic.com/docs/logs/forward-logs/circle
    ci-logs/
    How to set up Datadog / New Relic integration

    View full-size slide

  39. 46
    CircleCI Server

    View full-size slide

  40. 47
    CircleCI Server
    Cloud
    Developers / users
    VCS
    (GitHub.com or
    Bitbucket Cloud)
    Database
    Build fleet
    Cache and
    artifacts
    Server
    Within the client’s network,
    CircleCI dedicated for the client is set up
    Developers / users
    VCS
    (GitHub.com or GitHub
    Enterprise Server)
    Database
    Build fleet
    Cache and
    artifacts
    CircleCI will set up instances and work on security
    and maintenance operations

    View full-size slide

  41. 49
    CircleCI Server - Supported VCS
    Supported VCS
    VCS Cloud Server
    GitHub.com ✔ ✔
    GitHub
    Enterprise Server
    ✗ ✔
    Bitbucket Cloud ✔ ✗
    Bitbucket Server ✗ ✗

    View full-size slide

  42. 50
    CircleCI Server - Architecture Summary
    “Service cluster”
    (UI provision, authentication, and
    orchestration)
    Nomad cluster (Docker),
    VM Service,
    Runners
    (Actual job execution)
    VCS
    End users

    View full-size slide

  43. 52
    CircleCI Strength
    Robust feature-set for
    optimization and
    developer efficiency
    Quickly and confidently
    validate any code changes
    across your projects.
    Data and insights for
    better informed decisions
    Monitor and improve your
    team’s tests.
    Platform performance
    offers fastest route to
    deploy
    Quickly and confidently
    validate any code changes
    across your projects.

    View full-size slide

  44. 53
    Strong features to improve development velocity
    ● SSH debugging
    ○ Securely access any job on CircleCI to debug builds and tests in
    real-time.
    ● Insights Dashboard
    ○ Insights allow team to use data to improve engineering
    productivity
    ● Execution environment & fleet options
    ○ Choose from a vast fleet of executor types:
    Linux, Arm, macOS, & Windows.
    All offer a range of CPUs, GPUs, memory, and
    images to customize each job.
    ● Test splitting
    ○ Shorten the feedback loop by automatically
    splitting your tests across parallel instances of
    the same job

    View full-size slide

  45. 54
    ● Self-hosted runners(CircleCI runner)
    ○ Expand your compute options to meet your most unique CI/CD
    needs
    ● CircleCI Server
    ○ The power of CircleCI, on-prem or in your private cloud
    ● IP Ranges
    ○ Teams are able to open up their IP-based
    firewalls to only CircleCI
    ● Webhook
    ○ Teams are able to receive information (referred
    as events) from CircleCI, as they happen
    Other CircleCI advanced features

    View full-size slide

  46. 56
    OpenID Connect

    View full-size slide

  47. 57
    Authentication to cloud providers using static credentials

    View full-size slide

  48. 58
    Authentication to cloud providers using OpenID Connect

    View full-size slide

  49. 59
    Authentication to AWS using OpenID Connect

    View full-size slide

  50. 60
    Authentication to AWS using OpenID Connect
    (CircleCI Orb)
    https://github.com/tadashi0713/circleci-graviton-fargate-demo
    https://circleci.com/developer/orbs/orb/circleci/aws-ecr

    View full-size slide