Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Test-automation and CI in practice

Test-automation and CI in practice

This is an updated version of the "TDD in the real world" talk I did in 2015 for Reykjavík University, original available here: https://speakerdeck.com/steinnes/tdd-in-the-real-world

Steinn Eldjárn Sigurðarson

October 31, 2016
Tweet

More Decks by Steinn Eldjárn Sigurðarson

Other Decks in Technology

Transcript

  1. QuizUp 1.0 • Few client tests • A lot of

    server tests • Server tests often too large and slow • .. or worse: flaky • Fairly simple app
  2. QuizUp 1.0 • Several server deployments every single day •

    Client releases were a lot more scary • “Bricked” ~30k installs with one release
  3. QuizUp 2.0 • Excellent test coverage of clients • Total

    rewrite — it was hard, but it worked! • An order of magnitude more complex client • Maintained development velocity
  4. Different types of tests • Unit tests? • Integration tests?

    • System tests? • End-to-end tests? • … the evil ice cream!?
  5. Different types of tests • Usually better to test the

    smallest possible units • Small tests are fast, rarely break and are run often • Large tests with many “moving parts” are slower and tend to break more often due to external factors: • network issues • race conditions • random / rare factors
  6. Takumi Backend • Then: • ~5100 LOC vs. ~4400 LOC

    of tests • 35 seconds to run entire test suite • Unit tests run in 2 seconds and cover 57% • Integration tests in 30-35 seconds, cover 84% • Combined coverage: 91% • 97 unit tests, 258 integration tests
  7. Takumi Backend • Now: • ~10400 LOC vs. ~10100 LOC

    of tests • Server tests run in 2-3 minutes • Unit tests take 15 seconds and cover 78% • Integration tests 2-3 minutes and cover 68% • Combined coverage: 89% • 570 unit tests, 242 integration tests
  8. Takumi Backend • 1652 commits since launch • 337 ->

    849 pull requests • hundreds of deployments • 109 migrations
  9. Takumi App(s) • Timeline • April 2015 - October 2015:

    0.x.x • October 2015 - June 2016: 1.x.x • June 2016 - now: 2.x.x
  10. Takumi App(s) • 0.x.z had some tests. The bad kind.

    • 1.x.z had no tests, was written because 0.x.z was unstable • 2.x.z has tests, bugs are TDD’d, still working on optimal test patterns for RN components
  11. To TDD or not to TDD? • Often yields clear,

    concise and testable code • It took me ~8 years to understand the role of tests • You can go far without ever TDD-ing • When in doubt, alternate between coding and testing • Think of test code as the code which exposes the clarity, or lack thereof, of your real code
  12. To TDD or not to TDD? • If you end

    up with a huge suite of tests asserting minor interactions with a huge mock: no • Think of TDD-ing and RG-refactoring like weightlifting: • They build core strength which comes in handy during everyday life, improves your performance in your activity of choice and reduces the chance of injury. • (if done correctly!)
  13. CI / CD • What’s the purpose of CI? •

    Automate test runs to prevent regressions • Increase confidence for deployment or release
  14. CD? • What’s the purpose of CD? • Brilliant for

    shared development/staging environments • When done for large scale systems, rollout is usually slow and easily abortable • Great for development velocity, challenging for service reliability