Test-automation and CI in practice

This is an updated version of the "TDD in the real world" talk I did in 2015 for Reykjavík University, original available here: https://speakerdeck.com/steinnes/tdd-in-the-real-world

Transcript

1. Test-automation and CI in practice Steinn Eldjárn Sigurðarson
2. What is software engineering?
3. Why do we test? 3
4. Why do we test? • Not to prove correctness 4
5. Why do we test? • To facilitate changes
6. Why do we test? • To increase confidence
7. When do we test? 7
8. When do we test? • Small projects?
9. When do we test? • LARGE projects?
10. Story: QuizUp 1.0 -> 2.0
11. QuizUp 1.0 • Few client tests • A lot of server tests • Server tests often too large and slow • .. or worse: flaky • Fairly simple app
12. QuizUp 1.0 • Several server deployments every single day • Client releases were a lot more scary • “Bricked” ~30k installs with one release
13. QuizUp 2.0 • Huge set of new features • Massively more complex app!
16. QuizUp 2.0 • Excellent test coverage of clients • Total rewrite — it was hard, but it worked! • An order of magnitude more complex client • Maintained development velocity
17. Different types of tests • Unit tests? • Integration tests? • System tests? • End-to-end tests? • … the evil ice cream!?
19. Different types of tests • Usually better to test the smallest possible units • Small tests are fast, rarely break and are run often • Large tests with many “moving parts” are slower and tend to break more often due to external factors: • network issues • race conditions • random / rare factors
20. Different types of tests • What happens if you write large/flaky tests, well..
21. Different types of tests http://imgur.com/j11G35h
22. Story: Takumi
23. Platform for Influencer Campaigns
25. Takumi Backend • Then: • ~5100 LOC vs. ~4400 LOC of tests • 35 seconds to run entire test suite • Unit tests run in 2 seconds and cover 57% • Integration tests in 30-35 seconds, cover 84% • Combined coverage: 91% • 97 unit tests, 258 integration tests
26. Takumi Backend • Now: • ~10400 LOC vs. ~10100 LOC of tests • Server tests run in 2-3 minutes • Unit tests take 15 seconds and cover 78% • Integration tests 2-3 minutes and cover 68% • Combined coverage: 89% • 570 unit tests, 242 integration tests
27. Takumi Backend • 1652 commits since launch • 337 -> 849 pull requests • hundreds of deployments • 109 migrations
28. Takumi App(s) • Timeline • April 2015 - October 2015: 0.x.x • October 2015 - June 2016: 1.x.x • June 2016 - now: 2.x.x
29. Takumi App(s) • 0.x.z had some tests. The bad kind. • 1.x.z had no tests, was written because 0.x.z was unstable • 2.x.z has tests, bugs are TDD’d, still working on optimal test patterns for RN components
30. TDD ALL THE THINGS?
31. Nope. Not really.
32. To TDD or not to TDD? • Often yields clear, concise and testable code • It took me ~8 years to understand the role of tests • You can go far without ever TDD-ing • When in doubt, alternate between coding and testing • Think of test code as the code which exposes the clarity, or lack thereof, of your real code
33. To TDD or not to TDD? • If you end up with a huge suite of tests asserting minor interactions with a huge mock: no • Think of TDD-ing and RG-refactoring like weightlifting: • They build core strength which comes in handy during everyday life, improves your performance in your activity of choice and reduces the chance of injury. • (if done correctly!)
34. CI / CD • What’s the purpose of CI? • Automate test runs to prevent regressions • Increase confidence for deployment or release
35. CD? • What’s the purpose of CD? • Brilliant for shared development/staging environments • When done for large scale systems, rollout is usually slow and easily abortable • Great for development velocity, challenging for service reliability
36. Example CI pipeline
37. The Takumi “pipeline”
38. Thank you!