Reasonable Auto Scaling CI with AWS Spot Instance

D203fca278749fe5caa0338143f635cc?s=47 YutaTetsuka
October 13, 2017

Reasonable Auto Scaling CI with AWS Spot Instance

Gitlab CIとAWS Spot Instanceを使ってオートスケールなCIを実現しよう

D203fca278749fe5caa0338143f635cc?s=128

YutaTetsuka

October 13, 2017
Tweet

Transcript

  1. 1.

    Reasonable Auto Scaling CI with AWS Spot Instance GitLab Meetup

    Tokyo #4 Yuta Tetsuka 2017-10-12 Last update: 2017-10-13
  2. 2.

    Yuta Tetsuka - JustSystems corporation joined in 2012 - Team

    smilezemi (smile-zemi.jp) - Android engineer - Qiita: @tetsukay 1 About me 2
  3. 3.

    3

  4. 6.

    Before CI Environments - Gitlab CI - AWS on-demand Instance

    - m4.xlarge - 4core 16GB $0.258 / h 6
  5. 7.

    Problems PS: 2017-10-13 - CI is slow - Avg 26min

    - Can't concurrent build - 2 builds… 55min - 4 builds… 180min (timeout) - High Cost - 72 USD / month - 280 hours / month - (It was not as good as I thought it was.) 7
  6. 11.

    Auto scaling GitLab CI - Use AWS Spot Instance -

    by docker-machine - Create Instance per CI 11
  7. 14.

    Spot Instance merit - Low cost - about 15%〜20% of

    on-demand price - e.g. c4.4xlarge - On-demand: $1.008 / h - Spot: $0.182 / h - per-second billing model 14
  8. 17.

    Spot Instance demerit 2 - Shut down if it soars

    above the bid price - Time required to start instance - about 1〜2 min 17
  9. 18.

    After CI Environments - Gitlab CI Auto Scaling - AWS

    Spot Instance - c4.4xlarge - 16core 30GB $0.182 / h (minimum) - Bid $0.5 - CI Time 25min -> 14min - Unlimited Concurrent CI!! - PS 2017-10-13: (until aws limits, normally 5 concurrents) - PS 2017-10-13: My case is 20 18
  10. 20.

    Runner - Instance Type - t2.micro (1core 1GB) - 1.7

    Yen / h - Executor - docker+machine 20
  11. 21.

    Install docker-machine # Mac $ curl -L https://github.com/docker/machine/releases/download/v0.12.2/docker-machine-`uname -s`-`uname -m`

    >/usr/local/bin/docker-machine && \ chmod +x /usr/local/bin/docker-machine # Windows with Git bash $ if [[ ! -d "$HOME/bin" ]]; then mkdir -p "$HOME/bin"; fi && \ curl -L https://github.com/docker/machine/releases/download/v0.12.2/docker-machine-Windows-x86 _64.exe > "$HOME/bin/docker-machine.exe" && \ chmod +x "$HOME/bin/docker-machine.exe" # Linux $ curl -L https://github.com/docker/machine/releases/download/v0.12.2/docker-machine-`uname -s`-`uname -m` >/tmp/docker-machine && chmod +x /tmp/docker-machine && sudo cp /tmp/docker-machine /usr/local/bin/docker-machine 21
  12. 22.

    Test docker-machine docker-machine create \ --driver amazonec2 \ --amazonec2-region ap-northeast-1

    \ --amazonec2-request-spot-instance \ --amazonec2-spot-price 0.05 \ --amazonec2-ami=ami-2a69be4c \ --amazonec2-instance-type m4.large \ --amazonec2-private-address-only \ --amazonec2-vpc-id vpc-xxxxx \ --amazonec2-subnet-id subnet-xxxxx \ test-instance-name 22 Result
  13. 23.

    Register Runner gitlab-runner register --non-interactive \ --url https://gitlab.example.com/ci \ --registration-token

    [TOKEN] \ --executor "docker+machine" \ --name "gitlab-ci-auto-scaling" \ --docker-image "ubuntu" \ --machine-machine-driver "amazonec2" \ --machine-machine-name "gitlab-ci-%s" \ --machine-machine-options "amazonec2-ami=ami-2a69be4c" \ --machine-machine-options "amazonec2-region=ap-northeast-1" \ --machine-machine-options "amazonec2-request-spot-instance" \ --machine-machine-options "amazonec2-use-ebs-optimized-instance=true" \ --machine-machine-options "amazonec2-instance-type=c4.4xlarge" \ --machine-machine-options "amazonec2-spot-price=0.20" \ --machine-machine-options "amazonec2-vpc-id=[IF NEED]" \ --machine-machine-options "amazonec2-subnet-id=[IF NEED]" \ --machine-machine-options "amazonec2-ssh-keypath-id=[IF NEED]" \ --machine-machine-options "amazonec2-keypair-name=[IF NEED]" 23
  14. 25.
  15. 26.

    After - Runner Instance… - t2.micro $0.0152 - 280 hours

    / month - 280 * $0.0152 = $4.256 - CI Instances - c4.4xlarge $0.1825 (minimum) - (about) 200 CI / month - 14 min * 200 = 2800 min / month - 46.67 hours * (about) $0.2 = $9.33 26
  16. 27.

    Conclusion - c4.4xlarge is powerful ! - We could upgrade

    CI machine specs - Unlimited concurrent build ! - Very low cost ! 27
  17. 35.

    True-Up Modelとは? - 公式の説明が少しわかりにくい 35 The True-Up model seems complicated,

    can you illustrate? If you have 100 active users today, you should purchase a 100 user subscription. Suppose that when you renew next year you have 300 active users (200 extra users). When you renew you pay for a 300 user subscription and you also pay the full annual fee for the 200 users that you added during the year.