CIE as SaaS. Why is Apiary using CircleCI?

Slide 1

Slide 1 text

CIE as SaaS Why is Apiary using CircleCI? Ladislav Prskavec - HiveTalks, 9.1. 2019 1

Slide 2

Slide 2 text

Apiary 4 CircleCI, TravisCI, AppVeyor and Wercker Ladislav Prskavec - HiveTalks, 9.1. 2019 2

Slide 3

Slide 3 text

Timeline 4 2012 Started using CircleCI 4 2015 Parallel testing (initially 2, now 8 nodes) 4 2016 Docker 4 2017 CircleCI 2.0 (docker native) Ladislav Prskavec - HiveTalks, 9.1. 2019 3

Slide 4

Slide 4 text

What do you need from your CIE? Ladislav Prskavec - HiveTalks, 9.1. 2019 4

Slide 5

Slide 5 text

What do you need from your CIE? 1. Fast 2. Reliable 3. Scalable 4. Maintenance costs Ladislav Prskavec - HiveTalks, 9.1. 2019 5

Slide 6

Slide 6 text

1. Fast Ladislav Prskavec - HiveTalks, 9.1. 2019 6

Slide 7

Slide 7 text

Fast 4 Do you know your time to deploy? 4 Do you know how much time is spent in queue? 4 What is the maximum acceptable time for your build (5, 15, 30 min, 4h)? 4 Do you monitor those times or just guessing? Ladislav Prskavec - HiveTalks, 9.1. 2019 7

Slide 8

Slide 8 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 8

Slide 9

Slide 9 text

How we solved this problem? Ladislav Prskavec - HiveTalks, 9.1. 2019 9

Slide 10

Slide 10 text

4 Awareness (dashboards, metrics) 4 Groups (master, nightly) 4 Parallelism (speedup tests) Ladislav Prskavec - HiveTalks, 9.1. 2019 10

Slide 11

Slide 11 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 11

Slide 12

Slide 12 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 12

Slide 13

Slide 13 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 13

Slide 14

Slide 14 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 14

Slide 15

Slide 15 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 15

Slide 16

Slide 16 text

2. Reliable Ladislav Prskavec - HiveTalks, 9.1. 2019 16

Slide 17

Slide 17 text

4 Detect flaky tests (2 builds with 1 git hash) 4 Awareness (dashboards, metrics) 4 Process 4 Consistency Ladislav Prskavec - HiveTalks, 9.1. 2019 17

Slide 18

Slide 18 text

Retries in 2018 Ladislav Prskavec - HiveTalks, 9.1. 2019 18

Slide 19

Slide 19 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 19

Slide 20

Slide 20 text

Process Ladislav Prskavec - HiveTalks, 9.1. 2019 20

Slide 21

Slide 21 text

1. Responsibility Every test and helper needs to be clearly assigned to some team so in case of issues the team can be reached. Ladislav Prskavec - HiveTalks, 9.1. 2019 21

Slide 22

Slide 22 text

2. Maintenance There's a team responsible for every test, so flaky and long running tests can and should be a part of the sprint work. Ladislav Prskavec - HiveTalks, 9.1. 2019 22

Slide 23

Slide 23 text

Consistency Ladislav Prskavec - HiveTalks, 9.1. 2019 23

Slide 24

Slide 24 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 24

Slide 25

Slide 25 text

3. Scalable Ladislav Prskavec - HiveTalks, 9.1. 2019 25

Slide 26

Slide 26 text

4 How long are people waiting for CI results? 4 Can you run your tests in parallel? 4 Can you add 10 developers into team and time to deploy will stay the same? 4 How much does your build infrastructure cost? 4 Do you have budget for build/test infrastructure? Ladislav Prskavec - HiveTalks, 9.1. 2019 26

Slide 27

Slide 27 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 27

Slide 28

Slide 28 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 28

Slide 29

Slide 29 text

Ladislav Prskavec - HiveTalks, 9.1. 2019 29

Slide 30

Slide 30 text

4. Maintenance costs Ladislav Prskavec - HiveTalks, 9.1. 2019 30

Slide 31

Slide 31 text

Operations costs 4 SaaS (Circle, flat fee: $50 per container, dedicated support) 4 OnPremise (Jenkins master + workers) 4 Hybrid (Wercker, use managed K8S cluster for workers) 4 Combinations (JenkinsX, Gitlab, ...) Ladislav Prskavec - HiveTalks, 9.1. 2019 31

Slide 32

Slide 32 text

Summary 1. Fast 2. Reliable 3. Scalable 4. Maintenance costs 5. Monitoring Ladislav Prskavec - HiveTalks, 9.1. 2019 32

Slide 33

Slide 33 text

Q & A Ladislav Prskavec - HiveTalks, 9.1. 2019 33