CIE as SaaS. Why is Apiary using CircleCI?

# What do you need from your CIE?

1. Fast
2. Reliable
3. Scalable
4. Maintenance costs
5. Monitoring

Transcript

1. CIE as SaaS
   Why is Apiary using
   CircleCI?
   Ladislav Prskavec - HiveTalks, 9.1. 2019 1
   

   View Slide

2. Apiary
   4 CircleCI, TravisCI, AppVeyor and Wercker
   Ladislav Prskavec - HiveTalks, 9.1. 2019 2
   

   View Slide

3. Timeline
   4 2012 Started using CircleCI
   4 2015 Parallel testing (initially 2, now 8 nodes)
   4 2016 Docker
   4 2017 CircleCI 2.0 (docker native)
   Ladislav Prskavec - HiveTalks, 9.1. 2019 3
   

   View Slide

4. What do you need from
   your CIE?
   Ladislav Prskavec - HiveTalks, 9.1. 2019 4
   

   View Slide

5. What do you need from your CIE?
   1. Fast
   2. Reliable
   3. Scalable
   4. Maintenance costs
   Ladislav Prskavec - HiveTalks, 9.1. 2019 5
   

   View Slide

6. 1. Fast
   Ladislav Prskavec - HiveTalks, 9.1. 2019 6
   

   View Slide

7. Fast
   4 Do you know your time to deploy?
   4 Do you know how much time is spent in queue?
   4 What is the maximum acceptable time for your build
   (5, 15, 30 min, 4h)?
   4 Do you monitor those times or just guessing?
   Ladislav Prskavec - HiveTalks, 9.1. 2019 7
   

   View Slide

8. Ladislav Prskavec - HiveTalks, 9.1. 2019 8
   

   View Slide

9. How we solved this
   problem?
   Ladislav Prskavec - HiveTalks, 9.1. 2019 9
   

   View Slide

10. 4 Awareness (dashboards, metrics)
    4 Groups (master, nightly)
    4 Parallelism (speedup tests)
    Ladislav Prskavec - HiveTalks, 9.1. 2019 10
    

    View Slide

11. Ladislav Prskavec - HiveTalks, 9.1. 2019 11
    

    View Slide

12. Ladislav Prskavec - HiveTalks, 9.1. 2019 12
    

    View Slide

13. Ladislav Prskavec - HiveTalks, 9.1. 2019 13
    

    View Slide

14. Ladislav Prskavec - HiveTalks, 9.1. 2019 14
    

    View Slide

15. Ladislav Prskavec - HiveTalks, 9.1. 2019 15
    

    View Slide

16. 2. Reliable
    Ladislav Prskavec - HiveTalks, 9.1. 2019 16
    

    View Slide

17. 4 Detect flaky tests (2 builds with 1 git hash)
    4 Awareness (dashboards, metrics)
    4 Process
    4 Consistency
    Ladislav Prskavec - HiveTalks, 9.1. 2019 17
    

    View Slide

18. Retries in 2018
    Ladislav Prskavec - HiveTalks, 9.1. 2019 18
    

    View Slide

19. Ladislav Prskavec - HiveTalks, 9.1. 2019 19
    

    View Slide

20. Process
    Ladislav Prskavec - HiveTalks, 9.1. 2019 20
    

    View Slide

21. 1. Responsibility
    Every test and helper needs to be clearly assigned to
    some team so in case of issues the team can be reached.
    Ladislav Prskavec - HiveTalks, 9.1. 2019 21
    

    View Slide

22. 2. Maintenance
    There's a team responsible for every test, so flaky and
    long running tests can and should be a part of the
    sprint work.
    Ladislav Prskavec - HiveTalks, 9.1. 2019 22
    

    View Slide

23. Consistency
    Ladislav Prskavec - HiveTalks, 9.1. 2019 23
    

    View Slide

24. Ladislav Prskavec - HiveTalks, 9.1. 2019 24
    

    View Slide

25. 3. Scalable
    Ladislav Prskavec - HiveTalks, 9.1. 2019 25
    

    View Slide

26. 4 How long are people waiting for CI results?
    4 Can you run your tests in parallel?
    4 Can you add 10 developers into team and time to
    deploy will stay the same?
    4 How much does your build infrastructure cost?
    4 Do you have budget for build/test infrastructure?
    Ladislav Prskavec - HiveTalks, 9.1. 2019 26
    

    View Slide

27. Ladislav Prskavec - HiveTalks, 9.1. 2019 27
    

    View Slide

28. Ladislav Prskavec - HiveTalks, 9.1. 2019 28
    

    View Slide

29. Ladislav Prskavec - HiveTalks, 9.1. 2019 29
    

    View Slide

30. 4. Maintenance costs
    Ladislav Prskavec - HiveTalks, 9.1. 2019 30
    

    View Slide

31. Operations costs
    4 SaaS (Circle, flat fee: $50 per container, dedicated
    support)
    4 OnPremise (Jenkins master + workers)
    4 Hybrid (Wercker, use managed K8S cluster for
    workers)
    4 Combinations (JenkinsX, Gitlab, ...)
    Ladislav Prskavec - HiveTalks, 9.1. 2019 31
    

    View Slide

32. Summary
    1. Fast
    2. Reliable
    3. Scalable
    4. Maintenance costs
    5. Monitoring
    Ladislav Prskavec - HiveTalks, 9.1. 2019 32
    

    View Slide

33. Q & A
    Ladislav Prskavec - HiveTalks, 9.1. 2019 33
    

    View Slide