Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[DjangoCon Europe 2016] To mock or not to mock, that is the question

[DjangoCon Europe 2016] To mock or not to mock, that is the question

Mocking is a very powerful testing concept that has some dangerous pitfalls. There are obvious use cases where mocks are an absolute requirement to be able to test a part of the app. Nevertheless sometimes apparently useful mocks can yield erroneous test results. This talk goes into deeper detail on the trade-off of using mocks in testing.

2a3082799c3df9a58d06bc1b81107752?s=128

Ana Balica

March 30, 2016
Tweet

Transcript

  1. T o
 
 or not to
 
 that is the

    question MO CK MO CK
  2. @anabalica a treatise narrated by
 
 from Potato of Londontowne.

    ANA BALICA
  3. Thou 
 shalt 
 write 
 tests. “ ” William

    Shakespeare
  4. Chapter one

  5. Mocks simulate
 the looks and behaviour 
 of real objects

  6. REAL fig.1 fig.2 MOCK

  7. mocks != stubs

  8. unittest.mock # Python 3.3 mock # Python 2.x

  9. Mock()

  10. Mock >>> from mock import Mock >>> m = Mock()

    >>> m.foo = 1 >>> m.foo 1 >>> m.bar <Mock name='mock.bar' id='4310136016'>
  11. Mock() MagicMock()

  12. __lt__ __gt__ __len__ __iter__ __bool__ __str__ __int__ __hash__ __exit__ __sizeof__

  13. Mock() MagicMock() patch()

  14. Patching from mock import patch with patch('rainbow.Pony') as MockPony: MockPony.return_value

    = 42
  15. Patching 'rainbow.Pony' # creatures.py class Pony: pass # rainbow.py from

    creatures import Pony pony = Pony()
  16. Chapter two

  17. Good mocks

  18. System calls with mock.patch.dict("foo.os.environ", {"LOCAL_ENV": "production"}): conf = Configuration() self.assertEqual(conf.env,

    "production") os.environ
  19. Streams @mock.patch('sys.stdout', new=StringIO) def test_silly_logger(self, mock_stdout): logger.print_refs() self.assertEqual(mock_stdout.getvalue(), 'Some refs')

    sys.stdout
  20. HTTP requests /responses requests.get with mock.patch("foo.requests.get") as mock_get: mock_get.return_value =

    mock.Mock(status_code=200) status = is_it_down(“http://djangocon.eu") self.assertFalse(status)
  21. IO operations m = mock.mock_open(read_data="DjangoCon \o/“) with mock.patch("foo.open", m, create=True):

    with open("bar.txt") as h: result = h.read() self.assertEqual(result, "DjangoCon \o/") open
  22. Clocks, time, timezones from django.utils import timezone tz = pytz.timezone('Asia/Tokyo')

    dt = datetime.datetime(2016, 1, 1, tzinfo=tz) with mock.patch.object(timezone, 'now', return_value=dt): print timezone.now() timezone.now
  23. Unpredictable results with mock.patch('foo.random.random', return_value=0.42): result = throw_dice() self.assertEqual(result, 0.42)

    random.random
  24. ✓ System calls ✓ Streams ✓ HTTP requests ✓ IO

    operations ✓ Clocks, time, timezones ✓ Unpredictable results
  25. Why we like them ✓ Save time ✓ Make impossible

    possible ✓ Exclude external dependencies
  26. Chapter three

  27. Bad mocks

  28. with mock.patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data) self.assertTrue(form.is_valid())

    saved_pony = form.save() self.assertTrue(saved_pony.age, 3) mock_save.assert_called_once() Problems?
  29. with mock.patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data) self.assertTrue(form.is_valid())

    saved_pony = form.save() self.assertTrue(saved_pony.age, 3) mock_save.assert_called_once() Problems?
  30. Problems? with mock.patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data)

    self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) mock_save.assert_called_once()
  31. Problems? with mock.patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data)

    self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) mock_save.assert_called_once()
  32. Problems? with mock.patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data)

    self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) mock_save.assert_called_twice()
  33. Problems? with mock.patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data)

    self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) mock_save.make_me_a_sandwich()
  34. ¯\_(ツ)_/¯

  35. Solution 1 with mock.patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony,

    data=data) self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) mock_save.assert_called_once_with()
  36. Solution 2 with mock.patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony,

    data=data) self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) self.assertEqual(mock_save.call_count, 1)
  37. Failure with mock.patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data)

    self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) self.assertEqual(mock_save.sandwich_count, 1)
  38. Solution 3 Test Driven Development

  39. Problems? with mock.patch.object(Pony, 'save_base') as mock_save: form = PonyForm(instance=pony, data=data)

    self.assertTrue(form.is_valid()) saved_pony = form.save() self.assertEqual(saved_pony.age, 3) self.assertEqual(mock_save.call_count, 1)
  40. Tests pass? SHIP IT!

  41. Maybe 
 it’s incomplete?

  42. None
  43. c = Client() response = c.post('/pony/', {'age': 1}) self.assertEqual(response.status_code, 201)

    Functional tests
  44. Unit tests Functional tests Mocks

  45. Conclusions

  46. Mocks
 can be
 dangerous

  47. Passing faulty tests give a false sense of security

  48. The end