Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CircleCIの高速化🚀 / CircleCI faster

CircleCIの高速化🚀 / CircleCI faster

【オフライン開催】Omotesando.rb #92
https://omotesandorb.connpass.com/event/302869/

Example code to skip tests if source trees are the same.
https://gist.github.com/sinsoku/7b4201787d36f8ef669abd38395a28db

Takumi Shotoku

December 07, 2023
Tweet

More Decks by Takumi Shotoku

Other Decks in Technology

Transcript

  1. CircleCIͷߴ଎Խ
    Omotesando.rb #92 2023/12/07
    1

    View full-size slide

  2. ࣗݾ঺հ
    • ໊લ: ਖ਼ಙ ޼(aka: ਆ଎)
    • GitHub: @sinsoku (ը૾ӈ্)
    • Twitter: @sinsoku_listy (ը૾ӈԼ)
    • ޷͖ͳݴޠ: Rust
    !
    , Ruby
    "
    • Railsྺ: 8೥͘Β͍
    2

    View full-size slide

  3. ࿩͢͜ͱ

    !
    CIΛ଎͘͢ΔΞΠσΞ

    "
    CircleCIͷ࢓༷ͱ੍໿


    ίʔυ঺հ
    !
    Rubyͷ࿩͸͋Γ·ͤΜ
    3

    View full-size slide

  4. !
    CIΛ଎͘͢ΔΞΠσΞ
    ιʔείʔυͷมߋ͕ͳ͍৔߹ɺςετΛ࣮ߦ͢Δඞཁ͕ͳ͍ɻ
    • fast-forward ͳίϛοτ
    • ίϛοτϝοηʔδ͚ͩมߋͨ͠ίϛοτ
    4

    View full-size slide

  5. ิ଍: fast-forward
    ӈਤͷϚʔδίϛοτ͸ࠨਤBͷιʔείʔυͱಉ͡ɻ
    5

    View full-size slide

  6. ྫ: rails/rails
    ࣮ࡍʹ rails/rails ͷϩάͰௐ΂ͯΈΔɻ
    ҎԼͷྫͩͱ f915f412c5 ͱ 64f57c2c9d ͕ಉ͡ιʔείʔυͰ
    ͢ɻ
    $ git log --oneline --graph -n 3
    * f915f412c5 (HEAD -> main) Merge pull request #49701 from eileencodes/use-custom-class-for-pending-migrations-connection
    |\
    | * 64f57c2c9d Use custom class for pending migrations connection
    |/
    * cae5bafbf2 Merge pull request #49700 from akhilgkrishnan/active-job-argument-typo
    |\
    6

    View full-size slide

  7. Tree object ͸ͲͪΒ΋ 0d51392
    $ git cat-file -p f915f412c5
    tree 0d513921cca36506f23afaaea4f252ca4563a1ab
    parent cae5bafbf2e74dba245e5afc0279a0aafd1ae156
    parent 64f57c2c9d7b356ebd120751171a0fadf8946e90
    author Eileen M. Uchitelle 1697735828 -0400
    committer GitHub 1697735828 -0400
    Merge pull request #49701 from eileencodes/use-custom-class-for-pending-migrations-connection
    $ git cat-file -p 64f57c2c9d
    tree 0d513921cca36506f23afaaea4f252ca4563a1ab
    parent cae5bafbf2e74dba245e5afc0279a0aafd1ae156
    author eileencodes 1697733331 -0400
    committer eileencodes 1697733713 -0400
    Use custom class for pending migrations connection
    7

    View full-size slide

  8. طʹଞͷํ΋ϒϩάʹॻ͍͍ͯΔɻ1
    1 https://songmu.jp/riji/entry/2021-03-08-utilize-git-tree-hash-in-testing.html
    8

    View full-size slide

  9. ͜ͷϒϩάͷख๏
    1. ςετ͕௨ͬͨπϦʔͷϋογϡ஋Λه࿥͢Δ
    • S3ʹϋογϡ஋ͷΦϒδΣΫτΛPUT
    2. ςετ࣮ߦલʹS3ΦϒδΣΫτΛGET
    • Ωʔ͕ଘࡏ͢Ε͹ςετΛεΩοϓ
    9

    View full-size slide

  10. ϒϩά͔ΒҾ༻
    - run:
    name: go test
    command: |
    repotree_hash=$(git rev-parse HEAD^{tree})
    set +e
    curl -f https://${treehash_bucket}.s3.amazonaws.com/${repotree_hash} 2>/dev/null
    status=$?
    set -e
    if [ $status = "0" ]; then
    echo 'skip testing because the test for the same source tree has already passed'
    exit 0
    fi
    go test ./...
    10

    View full-size slide

  11. ͜ͷख๏ͷ՝୊
    • Amazon S3͕ඞཁ
    • γΣϧεΫϦϓτͰεΩοϓ൑அΛ͍ͯ͠Δ
    • δϣϒ୯ҐͰεΩοϓͰ͖ͳ͍
    ͜ΕΒΛվળͰ͖ͳ͍͔ௐ΂ͨ
    11

    View full-size slide

  12. !
    CircleCIͷ࢓༷ͱ੍໿
    .circleci/config.yml ͷ࢓༷͸͜͜ʹશͯॻ͔Ε͍ͯΔɻ
    Configuration reference
    https://circleci.com/docs/configuration-reference/
    12

    View full-size slide

  13. ࢖͑ͦ͏ͳػೳΛ୳͢
    13

    View full-size slide

  14. The when Step 2
    jobs: # conditional steps may also be defined in `commands:`
    job_with_optional_custom_checkout:
    parameters:
    custom_checkout:
    type: string
    default: ""
    machine:
    image: ubuntu-2004:202107-02
    steps:
    - when:
    condition: <>
    steps:
    - run: echo "my custom checkout"
    - unless:
    condition: <>
    steps:
    - checkout
    2 https://circleci.com/docs/configuration-reference/#the-when-step
    14

    View full-size slide

  15. ͜Μͳײ͡Ͱ͍͚ͦ͏ʁ
    jobs:
    test:
    parameters:
    previous_state:
    type: string
    default: ""
    machine:
    image: ubuntu-2004:202107-02
    steps:
    - unless:
    condition:
    equal: [ success, << parameters.previous_state >> ]
    steps:
    - run: echo "Run Tests"
    - when:
    condition:
    equal: [ success, << parameters.previous_state >> ]
    steps:
    - run: echo "Skip Tests"
    15

    View full-size slide

  16. The when Step ͷݶք
    16

    View full-size slide

  17. ࣄલδϣϒʢεςοϓʣͷ஋͸࢖͑ͳ͍ 3
    3 GitHub Actions ͳΒ؆୯ʹ࣮ݱͰ͖Δ
    17

    View full-size slide

  18. Dynamic Configuration4
    version: 2.1
    setup: true
    orbs:
    continuation: circleci/[email protected]
    jobs:
    setup:
    executor: continuation/default
    steps:
    - checkout
    - run: ./generate-config > generated_config.yml
    - continuation/continue:
    configuration_path: generated_config.yml
    workflows:
    setup:
    jobs:
    - setup
    • CircleCI ͷ ҋ ศརͳػೳ
    • ੜ੒ͨ͠YAMLͰCIΛ࣮ߦ
    • ྫͰ͸ ./generate-config
    εΫϦϓτͰੜ੒
    4 https://circleci.com/docs/using-dynamic-configuration/
    18

    View full-size slide

  19. ͳΜͱ͔ͳΓͦ͏
    19

    View full-size slide


  20. ίʔυ঺հ
    # .circleci/config.yml
    version: 2.1
    setup: true
    parameters:
    tree_sha_path:
    type: string
    default: ".git_tree_sha"
    tree_status_path:
    type: string
    default: ".gh_tree_status.json"
    orbs:
    continuation: circleci/[email protected]
    gh: circleci/[email protected]
    commands:
    20

    View full-size slide

  21. save_previous_tree_status
    # ਌ίϛοτͷίϛοτεςʔλεΛGitHubͰऔಘͯ͠JSONʹอଘ͢Δɻ
    # ਌ίϛοτͷ tree sha ΛΩʔʹΩϟογϡʹอଘ͢Δɻ
    save_previous_tree_status:
    steps:
    - run:
    name: Save previous tree sha
    command: git rev-parse @^^{tree} > << pipeline.parameters.tree_sha_path >>
    # An access token with `statuses:read` permission must be set in the GITHUB_TOKEN environment variable.
    - run:
    name: Save previous commit status as tree status
    command: gh api "/repos/{owner}/{repo}/commits/$(git rev-parse @^)/status" > << pipeline.parameters.tree_status_path >>
    - save_cache:
    key: tree_status_{{ checksum "<< pipeline.parameters.tree_sha_path >>" }}
    paths:
    - << pipeline.parameters.tree_status_path >>
    - run:
    name: Remove previous files
    command: rm << pipeline.parameters.tree_sha_path >> << pipeline.parameters.tree_status_path >>
    21

    View full-size slide

  22. restore_current_tree_status
    # ݱίϛοτͷ tree sha ΛΩʔʹΩϟογϡΛ෮ݩ͢Δɻ
    restore_current_tree_status:
    steps:
    - run:
    name: Save current tree sha
    command: git rev-parse @^{tree} > << pipeline.parameters.tree_sha_path >>
    - restore_cache:
    key: tree_status_{{ checksum "<< pipeline.parameters.tree_sha_path >>" }}
    - run:
    name: Put empty file if tree status does not exist
    command: |
    if [ ! -e "<< pipeline.parameters.tree_status_path >>" ]; then
    touch << pipeline.parameters.tree_status_path >>
    fi
    22

    View full-size slide

  23. jobs:
    setup:
    executor: continuation/default
    steps:
    - checkout
    - gh/install
    - save_previous_tree_status
    - restore_current_tree_status
    - run:
    name: Embed previous status
    command: |
    PREVIOUS_STATE=$(cat << pipeline.parameters.tree_status_path >> | \
    jq -r '.statuses[] | select(.context == "ci/circleci: test") | .state')
    # .circleci/continue_config.yml ͷ `_PREVIOUS_STATE` Λஔ׵
    sed -i "s/_PREVIOUS_STATE/$PREVIOUS_STATE/" .circleci/continue_config.yml
    - run: cat .circleci/continue_config.yml
    - continuation/continue:
    configuration_path: .circleci/continue_config.yml
    workflows:
    setup:
    jobs:
    - setup
    23

    View full-size slide

  24. ϝΠϯͷYAML
    # .circleci/continue_config.yml
    version: 2.1
    parameters:
    previous_state:
    type: string
    default: "_PREVIOUS_STATE"
    jobs:
    test:
    docker:
    - image: cimg/base:stable
    steps:
    - checkout
    - unless:
    condition:
    equal: [ success, << pipeline.parameters.previous_state >> ]
    steps:
    - run: echo "Run tests"
    - when:
    condition:
    equal: [ success, << pipeline.parameters.previous_state >> ]
    steps:
    - run: echo "Skip tests"
    workflows:
    main:
    jobs:
    - test
    24

    View full-size slide

  25. !
    ಈ࡞֬ೝ
    25

    View full-size slide

  26. git commit --allow-empty Ͱςετ͕εΩοϓ
    26

    View full-size slide

  27. ࣮ࡍͷYAML͸gistࢀর5
    5 https://gist.github.com/sinsoku/7b4201787d36f8ef669abd38395a28db
    27

    View full-size slide

  28. !
    ͦͯ͠YAML৬ਓʹ...
    28

    View full-size slide