Achieving Resilient Code with Integration Tests

You are maybe like me: I never learned at school how to write tests. My teachers gave me at first a broad overview of computer history. Then, they explained me some basic design patterns. And to finish, I often had to write more or less basic programs, to validate and demonstrate my skills. Not the kind of code I would be really proud of today: the procrastinator monkey living in my head at this time was more thinking about planning my summer holidays, rather than writing Ninja code!

And to make things worse, my studies focused on network and system engineering. Not software architecture. Funny story, because I decided to become programmer a couple of years later…

What I realize now is that I don’t have as much time as before to learn. And in a world driven by business, where time is money, and where tradeoffs are the rule, there is rarely enough money to write both shiny new features and a complete test suite.

People who practice Test-Driven Development know how complicated it can be to write proper tests. TDD is often discouraging at first: the learning curve is steep. But this problem also exists in the testing world in general. Because writing good tests is hard, many beginners get headaches trying to reach this goal. How to convince project managers to have more time for writing tests in these conditions…

But “le jeu en vaut la chandelle” as we say in French ("the juice is worth the squeeze"). Well tested applications are not only easier to maintain and extend. They also have in general a better API. That’s what we will see in this talk, by focusing on how to write integration tests. Our journey will begin with a presentation of different testing strategies. We will then jump to the practical part, using Pytest, interface testing , dependency injections and stubs, amongst many others. And because we want to add nice buzzwords on our resume after PyConDE, we will finish this talk by automating the whole with Docker Compose.

[Talk given at PyConDE 2018]

Transcript

1. Achieving Resilient Code With Integration Tests Alexandre Figura · Site Reliability Engineer @ SysEleven GmbH Committing crimes on Github since 1999 (@arugifa)
2. Who Am I? 2 • Use Python with since 2014. • " Live in Berlin since 2016 $ • Always eat 2 desserts at lunch • Work with cool folks at SysEleven GmbH,
 best company in town.
3. Using Mocks Or how to live dangerously...
4. 4 Introduction
5. 4 Introduction Advantages Mocks are easy to write.
6. 4 Introduction Advantages Mocks are easy to write. Drawbacks - Tests become dependent on codebase implementation, - Refactoring implies more work/time, - Errors and changes in original code are hidden.
7. 4 Introduction Advantages Mocks are easy to write. Drawbacks - Tests become dependent on codebase implementation, - Refactoring implies more work/time, - Errors and changes in original code are hidden. *Unless you have a very good reason... Conclusion Never use Mocks*
8. Code Example on Github
9. 6 Why Do We Keep Using Mocks? Because we tend to be lazy. la·zy (lā′zē) adj. la·zi·er, la·zi·est Privileging short-term benefits over long-term maintenance.
10. 6 Why Do We Keep Using Mocks? Because we tend to be lazy. la·zy (lā′zē) adj. la·zi·er, la·zi·est Privileging short-term benefits over long-term maintenance. Short-term benefits: - Writing tests as fast as possible. - Not thinking about code design. - Reusing tons of mocking libraries. Long-term maintenance: - Refactoring codebase. - Replacing outdated libraries. - Fixing failing tests.
11. Please Do Not Try Mocks at Home
12. The Proper Way To write integration tests
13. 9 Basic Concepts Integration Tests are built around two ideas: 1. Dependency Injection, 2. Interface Testing. It sounds complicated. But it isn't.
14. 10 In Practice
15. 10 In Practice Tests Application Real
 External System Fake
 External System
16. 10 In Practice 1. You write fake objects, and inject them
 from your tests into your code. Tests Application Real
 External System Fake
 External System
17. 10 In Practice 1. You write fake objects, and inject them
 from your tests into your code. Tests Application Real
 External System Fake
 External System 2. When real objects are updated, you run interface tests to check that
 your fakes still behave accordingly.
18. Code Example on Github
19. 12 Summary Advantages: - Tests not dependent on implementation anymore, - Possible to change an injected dependency later on, - Refactoring becomes very smooth. Drawbacks: - Requires way more work at first.
20. Real World Examples When practice meets theory...
21. 14 To the Cloud and Beyond
22. 14 To the Cloud and Beyond Use Case We want to upload a static website To the Cloud Using OpenStack Object Store's API.
23. 14 To the Cloud and Beyond Use Case We want to upload a static website To the Cloud Using OpenStack Object Store's API. Container Object Object Object Object Object Store
24. 14 To the Cloud and Beyond Use Case We want to upload a static website To the Cloud Using OpenStack Object Store's API. Container Object Object Object Object Object Store Links: - Officiel API, - Fake Implementation, - Interface Tests, - Pytest Configuration.
25. 15 Alternatives Fake Injection + Interface Testing FI + Minimal Interface Testing Mock Injection Full Mocking More Work Less Coverage More Refactoring
26. 16 Minimalist Interface Testing Use Case: I have a bunch of holiday movies Stored on Blu-rays And need to convert them to MKV.
27. 16 Minimalist Interface Testing Use Case: I have a bunch of holiday movies Stored on Blu-rays And need to convert them to MKV. Problem: A Blu-Ray is 30-50GB large.
28. 16 Minimalist Interface Testing Use Case: I have a bunch of holiday movies Stored on Blu-rays And need to convert them to MKV. Problem: A Blu-Ray is 30-50GB large. Solution: Don't test Blu-ray conversion; only look for syntax errors.
29. 16 Minimalist Interface Testing Use Case: I have a bunch of holiday movies Stored on Blu-rays And need to convert them to MKV. Problem: A Blu-Ray is 30-50GB large. Solution: Don't test Blu-ray conversion; only look for syntax errors. Links: - Syntax Errors Testing, - Fake Implementations, - Dependency Injection.
30. 17 Mock Injection
31. 17 Mock Injection Too much work to write Fakes?
32. 17 Mock Injection Too much work to write Fakes? - Write a Mock-like test helper, - And inject it into your code as a dependency.
33. 17 Mock Injection Too much work to write Fakes? - Write a Mock-like test helper, - And inject it into your code as a dependency. Useful for simulating complex systems.
34. 17 Mock Injection Too much work to write Fakes? - Write a Mock-like test helper, - And inject it into your code as a dependency. Useful for simulating complex systems. Advantages: - Easy as mocks, - Tests remain independent of implementation
 (makes refactoring easier over the long-term).
35. Code Example on Github
36. Running Tests with
 Docker Compose And add resiliency to your tests
37. 20 Toolbox -Docker: to run the application with its system dependencies & -Docker Compose: to set-up the testing environment -Tox: to install the application with its Python dependencies -Pytest: to launch the tests
38. 21 Overview Tests independent of running platform. Working example available on Github.
39. 21 Overview Tests independent of running platform. Working example available on Github. Docker Compose
40. 21 Overview Tests independent of running platform. Working example available on Github. Docker Compose Application Container
41. 21 Overview Tests independent of running platform. Working example available on Github. Docker Compose Application Container Source Code Tox +
 Pytest
42. 21 Overview Tests independent of running platform. Working example available on Github. Docker Compose Application Container Source Code Tox +
 Pytest Database
 Container Other
 Container LDAP
 Container
43. Conclusion What you should remember
44. 23 Summary Mocks make refactoring discouraging. Interface Testing is more flexible, but involves more work. Dependency Injection will improve your code's API. Always ask yourself (or around), if you need 100% test coverage. And you will find out what kind of tests you have to write.
45. We're hiring! Interested? 
 Just say: "Hello!" Mail: jobs@syseleven.de WhatsApp, SMS: +49 171/8934073