Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Test-automation and CI in practice

Test-automation and CI in practice

This is an updated version of the "TDD in the real world" talk I did in 2015 for Reykjavík University, original available here: https://speakerdeck.com/steinnes/tdd-in-the-real-world

Steinn Eldjárn Sigurðarson

October 31, 2016
Tweet

More Decks by Steinn Eldjárn Sigurðarson

Other Decks in Technology

Transcript

  1. Test-automation and CI in
    practice
    Steinn Eldjárn Sigurðarson

    View full-size slide

  2. What is software
    engineering?

    View full-size slide

  3. Why do we test?
    3

    View full-size slide

  4. Why do we test?
    • Not to prove correctness
    4

    View full-size slide

  5. Why do we test?
    • To facilitate changes

    View full-size slide

  6. Why do we test?
    • To increase confidence

    View full-size slide

  7. When do we test?
    7

    View full-size slide

  8. When do we test?
    • Small projects?

    View full-size slide

  9. When do we test?
    • LARGE projects?

    View full-size slide

  10. Story:
    QuizUp 1.0 -> 2.0

    View full-size slide

  11. QuizUp 1.0
    • Few client tests
    • A lot of server tests
    • Server tests often too
    large and slow
    • .. or worse: flaky
    • Fairly simple app

    View full-size slide

  12. QuizUp 1.0
    • Several server deployments every single day
    • Client releases were a lot more scary
    • “Bricked” ~30k installs with one release

    View full-size slide

  13. QuizUp 2.0
    • Huge set of new features
    • Massively more complex app!

    View full-size slide

  14. QuizUp 2.0
    • Excellent test coverage of clients
    • Total rewrite — it was hard, but it worked!
    • An order of magnitude more complex client
    • Maintained development velocity

    View full-size slide

  15. Different types of tests
    • Unit tests?
    • Integration tests?
    • System tests?
    • End-to-end tests?
    • … the evil ice cream!?

    View full-size slide

  16. Different types of tests
    • Usually better to test the smallest possible units
    • Small tests are fast, rarely break and are run often
    • Large tests with many “moving parts” are slower and
    tend to break more often due to external factors:
    • network issues
    • race conditions
    • random / rare factors

    View full-size slide

  17. Different types of tests
    • What happens if you write large/flaky tests, well..

    View full-size slide

  18. Different types of tests
    http://imgur.com/j11G35h

    View full-size slide

  19. Story:
    Takumi

    View full-size slide

  20. Platform for Influencer Campaigns

    View full-size slide

  21. Takumi Backend
    • Then:
    • ~5100 LOC vs. ~4400 LOC of tests
    • 35 seconds to run entire test suite
    • Unit tests run in 2 seconds and cover 57%
    • Integration tests in 30-35 seconds, cover 84%
    • Combined coverage: 91%
    • 97 unit tests, 258 integration tests

    View full-size slide

  22. Takumi Backend
    • Now:
    • ~10400 LOC vs. ~10100 LOC of tests
    • Server tests run in 2-3 minutes
    • Unit tests take 15 seconds and cover 78%
    • Integration tests 2-3 minutes and cover 68%
    • Combined coverage: 89%
    • 570 unit tests, 242 integration tests

    View full-size slide

  23. Takumi Backend
    • 1652 commits since launch
    • 337 -> 849 pull requests
    • hundreds of deployments
    • 109 migrations

    View full-size slide

  24. Takumi App(s)
    • Timeline
    • April 2015 - October 2015: 0.x.x
    • October 2015 - June 2016: 1.x.x
    • June 2016 - now: 2.x.x

    View full-size slide

  25. Takumi App(s)
    • 0.x.z had some tests. The bad kind.
    • 1.x.z had no tests, was written because 0.x.z was
    unstable
    • 2.x.z has tests, bugs are TDD’d, still working on
    optimal test patterns for RN components

    View full-size slide

  26. TDD
    ALL THE THINGS?

    View full-size slide

  27. Nope. Not really.

    View full-size slide

  28. To TDD or not to TDD?
    • Often yields clear, concise and testable code
    • It took me ~8 years to understand the role of tests
    • You can go far without ever TDD-ing
    • When in doubt, alternate between coding and
    testing
    • Think of test code as the code which exposes the
    clarity, or lack thereof, of your real code

    View full-size slide

  29. To TDD or not to TDD?
    • If you end up with a huge suite of tests asserting
    minor interactions with a huge mock: no
    • Think of TDD-ing and RG-refactoring like weightlifting:
    • They build core strength which comes in handy
    during everyday life, improves your performance in
    your activity of choice and reduces the chance of
    injury.
    • (if done correctly!)

    View full-size slide

  30. CI / CD
    • What’s the purpose of CI?
    • Automate test runs to prevent regressions
    • Increase confidence for deployment or release

    View full-size slide

  31. CD?
    • What’s the purpose of CD?
    • Brilliant for shared development/staging
    environments
    • When done for large scale systems, rollout is
    usually slow and easily abortable
    • Great for development velocity, challenging for
    service reliability

    View full-size slide

  32. Example CI pipeline

    View full-size slide

  33. The Takumi “pipeline”

    View full-size slide