Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Test-automation in practice

Test-automation in practice

Updated version of my yearly talk about some real world testing / agile development examples and stories held as a guest at University of Reykjavík.

Steinn Eldjárn Sigurðarson

November 08, 2017
Tweet

More Decks by Steinn Eldjárn Sigurðarson

Other Decks in Technology

Transcript

  1. Test-automation and CI in
    practice
    Steinn Eldjárn Sigurðarson

    View full-size slide

  2. What is software
    engineering?

    View full-size slide

  3. Why do we test?
    3

    View full-size slide

  4. Why do we test?
    • Not to prove correctness
    4

    View full-size slide

  5. Why do we test?
    • To facilitate changes

    View full-size slide

  6. Why do we test?
    • To increase confidence

    View full-size slide

  7. QuizUp 1.0 -> 2.0

    View full-size slide

  8. QuizUp 1.0
    • Few client tests
    • A lot of server tests
    • Server tests often too
    large and slow
    • .. or worse: flaky
    • Fairly simple app

    View full-size slide

  9. QuizUp 1.0
    • Several server deployments every single day
    • Client releases were a lot more scary
    • “Bricked” ~30k installs with one release

    View full-size slide

  10. QuizUp 2.0
    • Huge set of new features
    • Massively more complex app!

    View full-size slide

  11. QuizUp 2.0
    • Excellent test coverage of clients
    • Total rewrite — it was hard, but it worked!
    • An order of magnitude more complex client
    • Maintained development velocity

    View full-size slide

  12. Platform for Influencer Campaigns

    View full-size slide

  13. Takumi Backend
    • At launch:
    • ~5100 LOC vs. ~4400 LOC of tests
    • 35 seconds to run entire test suite
    • Unit tests run in 2 seconds and cover 57%
    • Integration tests in 30-35 seconds, cover 84%
    • Combined coverage: 91%
    • 97 unit tests, 258 integration tests

    View full-size slide

  14. Takumi Backend
    • Two years later:
    • ~16300 LOC vs. ~11900 LOC of tests
    • Server tests run in 2-3 minutes
    • Unit tests take 24 seconds and cover 77%
    • Integration tests take 2 minutes and cover 63%
    • Combined coverage: 83%
    • 940 unit tests, 197 integration tests

    View full-size slide

  15. Takumi Backend
    • 1652 commits since launch
    • 337 -> 1334 pull requests
    • hundreds of deployments
    • 184 migrations

    View full-size slide

  16. Takumi Mobile App
    • Three from-scratch implementations in a year
    • First prototype written in 5 months
    • Rewritten in 2 months before launch
    • Rewritten in React-Native 6 months after launch

    View full-size slide

  17. Takumi Mobile App
    • First prototype had some tests
    • First rewrite had no tests, but a lot less code
    • Current version has tests, but we still rely on a lot
    of manual QA
    • Still learning how to test and TDD

    View full-size slide

  18. CI / CD
    • What’s the purpose of CI?
    • Automate test runs to prevent regressions
    • Increase confidence for deployment or release

    View full-size slide

  19. CD?
    • What’s the purpose of CD?
    • Brilliant for shared development/staging
    environments
    • When done for large scale systems, rollout is
    usually slow and easily abortable
    • Great for development velocity, challenging for
    service reliability

    View full-size slide

  20. Example CI pipeline

    View full-size slide

  21. The Takumi “pipeline”

    View full-size slide