Test-automation in practice

Transcript

1. Test-automation and CI in practice Steinn Eldjárn Sigurðarson

2. What is software engineering?

3. Why do we test? 3

4. Why do we test? • Not to prove correctness 4

5. Why do we test? • To facilitate changes

6. Why do we test? • To increase confidence

7. QuizUp 1.0 -> 2.0

8. QuizUp 1.0 • Few client tests • A lot of

   server tests • Server tests often too large and slow • .. or worse: flaky • Fairly simple app

9. QuizUp 1.0 • Several server deployments every single day •

   Client releases were a lot more scary • “Bricked” ~30k installs with one release

10. QuizUp 2.0 • Huge set of new features • Massively

    more complex app!
11. None
12. None

13. QuizUp 2.0 • Excellent test coverage of clients • Total

    rewrite — it was hard, but it worked! • An order of magnitude more complex client • Maintained development velocity

14. TAKUMI

15. Platform for Influencer Campaigns

16. None

17. Takumi Backend • At launch: • ~5100 LOC vs. ~4400

    LOC of tests • 35 seconds to run entire test suite • Unit tests run in 2 seconds and cover 57% • Integration tests in 30-35 seconds, cover 84% • Combined coverage: 91% • 97 unit tests, 258 integration tests

18. Takumi Backend • Two years later: • ~16300 LOC vs.

    ~11900 LOC of tests • Server tests run in 2-3 minutes • Unit tests take 24 seconds and cover 77% • Integration tests take 2 minutes and cover 63% • Combined coverage: 83% • 940 unit tests, 197 integration tests

19. Takumi Backend • 1652 commits since launch • 337 ->

    1334 pull requests • hundreds of deployments • 184 migrations

20. Takumi Mobile App • Three from-scratch implementations in a year

    • First prototype written in 5 months • Rewritten in 2 months before launch • Rewritten in React-Native 6 months after launch

21. Takumi Mobile App • First prototype had some tests •

    First rewrite had no tests, but a lot less code • Current version has tests, but we still rely on a lot of manual QA • Still learning how to test and TDD

22. CI / CD • What’s the purpose of CI? •

    Automate test runs to prevent regressions • Increase confidence for deployment or release

23. CD? • What’s the purpose of CD? • Brilliant for

    shared development/staging environments • When done for large scale systems, rollout is usually slow and easily abortable • Great for development velocity, challenging for service reliability

24. Example CI pipeline

25. The Takumi “pipeline”

26. Thank you!