Building a Python Cake Testing: The Layers of Your Application

Testing is a best practice for delivering reliable software, but it can be a hard subject when starting out. What should you test and why? How much testing is enough? So you spent three days and wrote out tests for everything in your module, but was that an effective use of your time?

This talk will give an overview of the different layers that you can write tests for and why you should have them. You start with unit tests, mix in some integration tests, and cover with acceptance tests. Sprinkle with specific testing tools to taste. Tools we’ll discuss include py.test, docker, and coverage. Although the talk focus will be on web apps, the ideas will be relevant to all Python applications.

Writing quality tests is important: flaky tests will cost more time than they save and filler tests that don’t test important areas will weigh you down over time. With stable and effective tests for all layers, you build code you can trust, that you can refactor quickly or change easily without breaking everything. It’s as easy as cake!

Transcript

1. Matt Behrens PyConUK 2016 Building a Python Cake Testing The Layers of Your Application
2. Yelp’s Mission Connecting people with great local businesses.
3. Yelp Stats As of Q2 2016 92M 32 72% 108M
4. Matt Behrens @askedrelic Who Am I?
5. “It’s done, I just have to write the tests” - Every Engineer
6. Tests Are Important!
8. [FAIL CAKE]
9. • The Testing Layer Cake • Python Recipes • Lessons and Tips Agenda
10. The Testing Layer Cake Smoke Tests
11. • Fast and consistent validation of one unit of code. • Build a solid foundation. Unit Tests Don’t: - network calls - database calls - side effects Do: - mocks
12. • AKA fakes, doubles, stubs, the fondant of testing • A tool for helping test one thing at a time. • Was the mock called? How many times? What was it called with? Mocks
13. @mock.patch('pybakery.oven') def test_oven_turns_on(mock_oven): mock_oven.turn_on = True the_oven = Oven() assert(the_oven.turn_on() == True)
14. • Test that your units connect together correctly. Test the relationships between your components. • Are you testing more than one class? • Mocks okay. ◦ Mock data. Mock network or DB calls! Integration Tests
15. • AKA system test, end to end tests • Test your application end to end, with real data, in a production like environment. • Try not to use mocks. Acceptance Tests
16. • Sometimes you can’t or won’t test something 100% ◦ What tradeoffs do you make to ship? ◦ How can you make it easier to test? • A chance to test in different environments for edge cases ◦ Used in Yelp deployment ▪ Verify in stage ▪ Verify in production with % of live traffic Manual Tests
17. • Simple check that everything is working and testing can continue ◦ Is the homepage on fire? • Generally blackbox and high level • Related: distributed monitoring is hard ◦ See Steve Yegge post about Amazon service architecture breakup Smoke Tests
18. Python Recipes
19. • Pytest ◦ Fixtures ◦ Parameterized test data ◦ Plugins • Mock ◦ Python 3.3+ builtin How To Unit and Integration Test
20. • Test one thing ◦ Anti-pattern is to expand your tests to cover multiple situations • Think about structure of a test ◦ Setup ◦ Test ◦ Validate How To Write Good Unit Tests
21. @pytest.fixture def sprinkles(): return ["°*°*°", '*~*~*~*~', '♥•**•**•♥'] def test_cake(sprinkles): cake = build_cake() # setup cake.bake(sprinkles) # test assert cake.tasty == True # validate
22. • Use as needed ◦ Don’t mock too much or too little ◦ Anti-pattern is having too many mocks • Check correct mock call_count ◦ Anti-pattern is using mock assert_called_once_with ◦ See Yelp Engineer blog post for more info How To Write Good Mocks
23. @pytest.yield_fixture def mock_bake(): with mock.patch( 'pybakery.cake.logic.oven.bake' ) as mock_fn: yield mock_fn def test_cake(mock_bake): ...
24. • Use a mirror structure pybakery/cake/logic/sprinkles.py tests/cake/logic/test_sprinkles.py How To Organize Unit Tests . ├─pybakery/cake/logic/oven.py └────tests/cake/logic/test_oven.py
25. • Docker ◦ Tool for running processes/environments in isolation ◦ Real MySQL process, real SQL data • Good ◦ Relatively easy to setup • Not so good ◦ Young project, changing rapidly ◦ Difficult to understand sometimes How To Acceptance Test
26. Lessons and Tips
27. • How much of your program is executed, when your (unit/integration) tests run • You probably don’t need 100% coverage! • Always good to know and improve on Code Coverage
28. • Is a global input! • Don’t forget to make it a function argument! You will want to override it. • Freezegun is library to mock time ◦ Supports Python 2, 3 Time
29. • Pattern for building a specific types of objects • Invest in making your testing easier def create(sprinkles=True, **kwargs): options.update(kwargs) Cake(**options) Factories def create(sprinkles=True, **kwargs): default_cake_options = {‘sprinkles’: sprinkles} default_cake_options.update(kwargs) Cake(**default_cake_options)
30. • Super easy to test ◦ Just inputs and outputs. Eliminate side effects! • Think about what layers you are building • Python Builtins ◦ import itertools, functools ◦ map(), filter(), reduce(), any(), all() ◦ List comprehensions Functional Programming
31. 1. Write a test first 2. Watch it fail 3. Write the logic to make it succeed 4. Refactor 5. Repeat • You know when you are done • You solve the right problem Test Driven Development
32. Each unit should have limited knowledge about other units. • Depend on interfaces, not objects. • Make classes easy to swap out. Law Of Demeter
33. How can you change the structure of your code to make it easier to test? Final Thoughts
34. Questions? @askedrelic mbehrens@yelp.com @YelpEngineering