Mock considered harmful

410e3353165c33043ab69be7fc366428?s=47 Boris Feld
October 18, 2015

Mock considered harmful

Why mock are often considered harmful and lead to bad test quality ? Is there other ways to test code without mocks?

410e3353165c33043ab69be7fc366428?s=128

Boris Feld

October 18, 2015
Tweet

Transcript

  1. 2.

    Let’s test def send_billing_email(): last_month = datetime.date.today() - datetime.timedelta(month=1) for

    user in USERS.find(is_active=True, last_paid__lt=last_month): UserEmailer.send(user, 'billing_reminder')
  2. 3.

    Let’s unit test it! Test UserEmailer Users • Faster to

    run • Easier to write • Devs are eager to write tests
  3. 4.

    How to test it? class TestBillingEmail(unittest.TestCase): @patch('code.USERS') @patch('code.UserEmailer') def test_send_billing_email(self,

    user_email_patch, users_patch): users = [sentinel.USER1, sentinel.USER2] users_patch.find.return_value = users code.send_billing_email() last_month = datetime.date.today() - datetime.timedelta(days=30) self.assertEqual(users_patch.find.call_args_list, [call(is_active=True, last_paid__lt=last_month)]) self.assertEqual(user_email_patch.send.call_args_list, [call(users[0], 'billing_reminder'), call(users[1], 'billing_reminder')])
  4. 5.

    Good? • Fast to run ✓ • Easy to write

    ✓ • Devs are eager to write tests ✓
  5. 7.

    Start refactoring • Change parameters order • Forbid a parameter

    value • Change output format • Add a new parameter • Add a new parameter with default value • Rename method
  6. 9.
  7. 10.

    Take a look back class TestBillingEmail(unittest.TestCase): @patch('code.USERS') @patch('code.UserEmailer') def test_send_billing_email(self,

    user_email_patch, users_patch): users = [sentinel.USER1, sentinel.USER2] users_patch.find.return_value = users code.send_billing_email() last_month = datetime.date.today() - datetime.timedelta(days=30) self.assertEqual(users_patch.find.call_args_list, [call(is_active=True, last_paid__lt=last_month)]) self.assertEqual(user_email_patch.send.call_args_list, [call(users[0], 'billing_reminder'), call(users[1], 'billing_reminder')])
  8. 11.
  9. 12.

    Test design • Test should not break if: • USERS

    changes its API • UserEmailer changes its API • => Change detector tests
  10. 16.

    - Wikipedia « In a unit test, mock objects can

    simulate the behavior of complex, real objects and are therefore useful when a real object is impractical or impossible to incorporate into a unit test. »
  11. 17.

    Four common usage of Mock • Test internal implementation, Cache

    system. • Simulate some weird errors, network errors. • Reduce randomness. • « Light-weight » test double for DB, Network, Disk.
  12. 19.

    Side-effect • As it test behaviors, it couples the test

    to the mocked object contract. • The test will fail every time the contract is changed. • The test may pass while real code breaks due to refactoring. • « White-box » testing with monkey patching.
  13. 22.

    Ideal black-box test • For a given « SUT »,

    given a set of input, we could assert the state of outputs. • => Functional programming
  14. 23.
  15. 24.

    - James Shore « Dependency injection means giving an object

    its instance variables. Really. That's it. »
  16. 25.

    Let’s try again def send_billing_email(user_model=USERS, user_emailer=UserEmailer): last_month = datetime.date.today() -

    datetime.timedelta(days=30) for user in user_model.find(is_active=True, last_paid__lt=last_month): user_emailer.send(user, 'billing_reminder') class TestBillingEmail(unittest.TestCase): def test_send_billing_email(self): users_mock = Mock() users = [sentinel.USER1, sentinel.USER2] users_mock.find.return_value = users user_emailer_mock = Mock() code.send_billing_email(users_mock, user_emailer_mock) last_month = datetime.date.today() - datetime.timedelta(days=30) self.assertEqual(users_mock.find.call_args_list, [call(is_active=True, last_paid__lt=last_month)]) self.assertEqual(user_emailer_mock.send.call_args_list, [call(users[0], 'billing_reminder'), call(users[1], 'billing_reminder')])
  17. 27.

    Design • The function need to get back filtered Users:

    • And send an Email to each user: user_model.find(is_active=True, last_paid__lt=last_month) user_emailer.send(user, 'billing_reminder')
  18. 28.

    Contract based dependencies • Think about replaceable dependencies • If

    it implements the contract… • We can interchange them • As if they were plugin or micro-services
  19. 29.

    Create a test per interface class _BaseUserModelTestCase(unittest.TestCase): def setUp(self): self.model

    = self.get_model() def test_insert(self): user = {'username': 'bfe'} self.model.insert(user) self.assertEqual(self.model.list(), [user]) def test_filter(self): users = [{'username': 'bfe', 'post': 42}, {'username': 'ohe', 'post': 25}] self.model.insert(users[0]) self.model.insert(users[1]) self.assertEqual(self.model.list(), users) self.assertEqual(self.model.find(username='bfe'), users[0]) self.assertEqual(self.model.find(post__lt=40), users[1])
  20. 30.

    Then a test per implementation class DBUserModelTestCase(_BaseUserModelTestCase): def get_model(self): return

    USERS class StubUserModelTestCase(_BaseUserModelTestCase): def get_model(self): return StubUserModel()
  21. 31.

    For reference, test implementation class StubUserModel(object): OPERATORS = { None:

    operator.eq, 'lt': operator.lt } def __init__(self): self.users = [] def insert(self, user): self.users.append(user) def list(self): return self.users def find(self, **kwargs): matching_users = [] for user in self.users: matching = True for kwarg, value in kwargs.items(): field = kwarg.split('__') if len(field) == 1: operator = None else: operator = field[1] if not self.OPERATORS[operator](user.get(field[0]), value): matching = False if matching: matching_users.append(user) return matching_users
  22. 32.

    And emailer class _BaseUserEmailerTestCase(unittest.TestCase): def setUp(self): self.user_emailer = self.get_user_emailer() def

    test_send(self): user = {'username': 'bfe'} topic = 'test_topic' self.user_emailer(user, topic) self.assertMailSent(user, topic) self.assertNotMailSent(user, topic)
  23. 34.

    Test double implementation class TestUserEmailer(object): def __init__(self): self.sent_emails = []

    def send(self, user, topic): self.sent_emails.append((user, topic)) def assertMailSent(self, user, topic): return (user, topic) in self.sent_emails def assertNotMailSent(self, user, topic): return not self.assertMailSent(user, topic)
  24. 35.

    And now? class TestBillingEmail(unittest.TestCase): def test_send_billing_email(self): model = StubUserModel() emailer

    = TestUserEmailer() # Fixtures one_month_ago = date.today() - timedelta(days=30) valid_user = {'username': 'bfe', 'is_active': True, 'last_paid': one_month_ago + timedelta(days=2)} inactive_user = {'username': 'foo', 'is_active': False, 'last_paid': one_month_ago - timedelta(days=2)} bad_payer_user = {'username': 'foo', 'is_active': True, 'last_paid': one_month_ago - timedelta(days=2)} for u in [valid_user, inactive_user, bad_payer_user]: model.insert(u) code.send_billing_email(model, emailer) emailer.assertMailSent(bad_payer_user, 'billing_reminder')
  25. 36.
  26. 37.

    Both models are worth it • Different needs • Mocks

    are cheaper • But stub requires less maintenance
  27. 38.

    Different design approachs • Mock is good for top-down approach

    • London-Style, mockist-style • Stubs is good for bottom-up approach • Chicago school, classic TDD • Both are complementary
  28. 39.

    Reflexion • « Interface » testing is good for design

    reflexion, how I want to call it and how the state is changed? • Helps to have modular design and clear contract.
  29. 40.

    Intermediary • Code call a proxy: • Which will forward

    to a real object for production • Or a STUB in tests • Ex: Django email backends / password backends • => django-nopassword