Test-automation in practice

Test-automation in practice

Updated version of my yearly talk about some real world testing / agile development examples and stories held as a guest at University of Reykjavík.

36f7b3921277b2ed27eb4798c18266e4?s=128

Steinn Eldjárn Sigurðarson

November 08, 2017
Tweet

Transcript

  1. Test-automation and CI in practice Steinn Eldjárn Sigurðarson

  2. What is software engineering?

  3. Why do we test? 3

  4. Why do we test? • Not to prove correctness 4

  5. Why do we test? • To facilitate changes

  6. Why do we test? • To increase confidence

  7. QuizUp 1.0 -> 2.0

  8. QuizUp 1.0 • Few client tests • A lot of

    server tests • Server tests often too large and slow • .. or worse: flaky • Fairly simple app
  9. QuizUp 1.0 • Several server deployments every single day •

    Client releases were a lot more scary • “Bricked” ~30k installs with one release
  10. QuizUp 2.0 • Huge set of new features • Massively

    more complex app!
  11. None
  12. None
  13. QuizUp 2.0 • Excellent test coverage of clients • Total

    rewrite — it was hard, but it worked! • An order of magnitude more complex client • Maintained development velocity
  14. TAKUMI

  15. Platform for Influencer Campaigns

  16. None
  17. Takumi Backend • At launch: • ~5100 LOC vs. ~4400

    LOC of tests • 35 seconds to run entire test suite • Unit tests run in 2 seconds and cover 57% • Integration tests in 30-35 seconds, cover 84% • Combined coverage: 91% • 97 unit tests, 258 integration tests
  18. Takumi Backend • Two years later: • ~16300 LOC vs.

    ~11900 LOC of tests • Server tests run in 2-3 minutes • Unit tests take 24 seconds and cover 77% • Integration tests take 2 minutes and cover 63% • Combined coverage: 83% • 940 unit tests, 197 integration tests
  19. Takumi Backend • 1652 commits since launch • 337 ->

    1334 pull requests • hundreds of deployments • 184 migrations
  20. Takumi Mobile App • Three from-scratch implementations in a year

    • First prototype written in 5 months • Rewritten in 2 months before launch • Rewritten in React-Native 6 months after launch
  21. Takumi Mobile App • First prototype had some tests •

    First rewrite had no tests, but a lot less code • Current version has tests, but we still rely on a lot of manual QA • Still learning how to test and TDD
  22. CI / CD • What’s the purpose of CI? •

    Automate test runs to prevent regressions • Increase confidence for deployment or release
  23. CD? • What’s the purpose of CD? • Brilliant for

    shared development/staging environments • When done for large scale systems, rollout is usually slow and easily abortable • Great for development velocity, challenging for service reliability
  24. Example CI pipeline

  25. The Takumi “pipeline”

  26. Thank you!