Ravi Chandra: Fake it till you make it

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Ravi Chandra:
Fake it till you make it
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2013 - Saturday, 07 Sep 2013 - Track 1
http://nz.pycon.org/

**Audience level**

Intermediate

**Description**

A practical introduction to unit testing decoupled, service-oriented, Python applications using with Mock library. A three-tiered (server, client, and view) Flask web application is used as a motivating example across the talk. Mock helps to isolate functionality specific to a layer to facilitate fast, compact, unit testing while avoiding writing specific mock classes or using fixtures.

**Abstract**

In practise most of us have found testing to be much harder to achieve in the "real world". That is often because the application architecture is modular with some level of service abstraction, and the ensuing layers of code.

The premise of this talk is: do not test someone else's code. They are a good developer. They would have tested it so it must work.

Taking this perspective allows us to develop fast and compact unit tests for our application functionality while ignoring bigger picture integration issues (these can be addresses using other types of tests!). In our experience this unburdens developers so they can really practice test-driven development much more diligently.

Implementation of this approach is simplified thanks to the mock library. Mock provides a core MagicMock class that obviates the need for custom stubs throughout the code. Moreover, its supports patching specific methods, properties, and more. Mocking is used to patch calls to external code within our unit tests, since we don't need to test them, in a non-non-invasive way to our application.

**YouTube**

http://www.youtube.com/watch?v=HbJZu2qcC-s

Transcript

1. Python Unit Testing with Mock
2. Fake It till you Make It Python Unit Testing with Mock Ravi Chandra & Ben Shaw
3. presentation overview • Testing: unit testing vs integration testing • Introduce an “enterprise” service-oriented EchoServer application as the motivating example • Practical exposition of the unit tests across the application, using the mock library
4. unit testing vs integration testing • Unit tests are written to verify a relatively small piece of code does what it is supposed to • Integration tests are written to verify the operation of a system as a whole, or the interaction between different pieces of the system
5. http://c2.com/cgi/wiki?TestDrivenDevelopment
6. Rules of Unit Testing 1.You do not test someone else’s code, they’ve tested it so it must work. 2.You do not talk about unit testing.
7. “Write me an application…” • That reads an input string, then, • Will print to the console either: • The input text • The input text, reversed • MD5 digest of the input text
8. from sys import argv from hashlib import md5 if argv[1] == 'echo': print ' '.join(argv[2:]) elif argv[1] == 'reverse': print ' '.join(argv[2:])[::-1] elif argv[1] == 'md5': print md5(' '.join(argv[2:])).hexdigest()
9. “not enterprisey enough.”
11. EchoView EchoClient EchoServer HTTP Python Echo Enterprise Architecture “service-oriented architecture” “separation of concerns”
12. EchoView EchoClient EchoServer HTTP Python Echo Enterprise Architecture
13. EchoView EchoClient EchoServer HTTP Python Echo Enterprise Architecture
14. EchoView EchoClient EchoServer HTTP Python Echo Enterprise Architecture
15. EchoView EchoClient EchoServer HTTP Python Echo Enterprise Architecture
16. EchoServer • Three endpoints: • /echo • /reverse • /md5 • Each takes a text value via POST, in value key.
17. from hashlib import md5 as md5_hash from flask import Flask, request app = Flask(__name__) @app.route("/echo", methods=['POST']) def echo(): return request.form['value'] @app.route("/reverse", methods=['POST']) def reverse(): return request.form['value'][::-1] @app.route("/md5", methods=['POST']) def md5(): return md5_hash(request.form['value']).hexdigest()
18. import unittest import echoserver class EchoServerTestCase(unittest.TestCase): def setUp(self): self.app = echoserver.app.test_client() def test_echo(self): # expect output to be identical to the input resp = self.app.post( '/echo', data=dict(value='test echo')) self.assertEqual(resp.data, 'test echo') def test_reverse(self): # expect output to be reversed input string resp = self.app.post( '/reverse', data=dict(value='test reverse')) self.assertEqual(resp.data, 'esrever tset') def test_md5(self): # expect output to be the md5 hash digest of input resp = self.app.post( '/md5', data=dict(value='MD5 Test Hash')) self.assertEqual( resp.data, '2273c77fba3d04d5c3eb8b9f309d258a')
19. EchoServer must work!
20. EchoView EchoClient EchoServer HTTP Python Echo Enterprise Architecture
21. EchoView EchoClient EchoServer HTTP Python Echo Enterprise Architecture
22. EchoClient • Three public methods: • echo() • reverse() • md5() • Each takes a single argument: the string to operate on. • Each returns the transformed string.
23. class EchoClient(object): def __init__(self, base_url): self.base_url = base_url def echo(self, value): return self._make_request('echo', value) def reverse(self, value): return self._make_request('reverse', value) def md5(self, value): return self._make_request('md5', value) def _build_url(self, endpoint): return urljoin(self.base_url, endpoint) def _make_request(self, endpoint, value): try: response = requests.post( self._build_url(endpoint), data=dict(value=value)) except RequestException as e: raise EchoClientError(e.__class__.__name__) return response.text
24. Whoever wrote EchoServer is a genius! her code works exactly as documented.
25. What do we need to test? • We know that the web service works as documented. • (See rule #1) • We just need to make sure we call the EchoServer correctly, in this case using the HTTP API.
26. mock to the rescue!
27. import unittest from mock import MagicMock, patch from echoclient import EchoClient, EchoClientError from requests.exceptions import HTTPError class EchoClientTestCase(unittest.TestCase): def setUp(self): self.api_url = 'http://test-base-url/' self.client = EchoClient(self.api_url) def test_base_url(self): # test the base URL is stored correctly self.assertEqual(self.client.base_url, self.api_url) def test_url_generation(self): # test we can generate API endpoint URLs self.assertEqual(self.client._build_url('endpoint'), 'http://test-base-url/endpoint') def test_echo(self): # test the 'echo' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.echo('echo test') self.client._make_request.assert_called_with('echo', 'echo test') def test_reverse(self): # test the 'reverse' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.reverse('reverse test') self.client._make_request.assert_called_with( 'reverse', 'reverse test') def test_md5(self): # test the 'md5' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.md5('md5 test') self.client._make_request.assert_called_with('md5', 'md5 test') @patch('requests.post') def test_make_request_call(self, mock_requests_post): # test _make_request actually makes the API call correctly with the given POST data and method data_value = 'an example data value' self.client._make_request('foo', data_value) mock_requests_post.assert_called_with( self.api_url + 'foo', data={'value': data_value}) def test_make_request_return_value(self): # test _make_request returns response content when successful resp = MagicMock() resp.status_code = 200 # unused for now! resp.text = 'return value' with patch('requests.post', return_value=resp) as a: data = self.client._make_request( 'foo', 'hello world') self.assertEqual(data, 'return value') @patch('requests.post', side_effect=HTTPError('eek!')) def test_requests_exception(self, mock_requests_post): # test _make_request propagates Requests errors as EchoClientErrors with self.assertRaises(EchoClientError) as e: self.client._make_request('foo', 'hello world') self.assertTrue( e.exception.message.startswith('HTTPError')) URL Generation API calls Request/ response Exception raising
28. import unittest from mock import MagicMock, patch from echoclient import EchoClient, EchoClientError from requests.exceptions import HTTPError class EchoClientTestCase(unittest.TestCase): def setUp(self): self.api_url = 'http://test-base-url/' self.client = EchoClient(self.api_url) def test_base_url(self): # test the base URL is stored correctly self.assertEqual(self.client.base_url, self.api_url) def test_url_generation(self): # test we can generate API endpoint URLs self.assertEqual(self.client._build_url('endpoint'), 'http://test-base-url/endpoint') def test_echo(self): # test the 'echo' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.echo('echo test') self.client._make_request.assert_called_with('echo', 'echo test') def test_reverse(self): # test the 'reverse' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.reverse('reverse test') self.client._make_request.assert_called_with( 'reverse', 'reverse test') def test_md5(self): # test the 'md5' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.md5('md5 test') self.client._make_request.assert_called_with('md5', 'md5 test') @patch('requests.post') def test_make_request_call(self, mock_requests_post): # test _make_request actually makes the API call correctly with the given POST data and method data_value = 'an example data value' self.client._make_request('foo', data_value) mock_requests_post.assert_called_with( self.api_url + 'foo', data={'value': data_value}) def test_make_request_return_value(self): # test _make_request returns response content when successful resp = MagicMock() resp.status_code = 200 # unused for now! resp.text = 'return value' with patch('requests.post', return_value=resp) as a: data = self.client._make_request( 'foo', 'hello world') self.assertEqual(data, 'return value') @patch('requests.post', side_effect=HTTPError('eek!')) def test_requests_exception(self, mock_requests_post): # test _make_request propagates Requests errors as EchoClientErrors with self.assertRaises(EchoClientError) as e: self.client._make_request('foo', 'hello world') self.assertTrue( e.exception.message.startswith('HTTPError')) API calls URL Generation Request/ response Exception raising
29. class EchoClientTestCase(unittest.TestCase): def setUp(self): self.api_url = 'http://test-base-url/' self.client = EchoClient(self.api_url) def test_base_url(self): # test the base URL is stored correctly self.assertEqual(self.client.base_url, self.api_url) def test_url_generation(self): # test we can generate API endpoint URLs self.assertEqual(self.client._build_url('endpoint'), 'http://test-base-url/endpoint')
30. import unittest from mock import MagicMock, patch from echoclient import EchoClient, EchoClientError from requests.exceptions import HTTPError class EchoClientTestCase(unittest.TestCase): def setUp(self): self.api_url = 'http://test-base-url/' self.client = EchoClient(self.api_url) def test_base_url(self): # test the base URL is stored correctly self.assertEqual(self.client.base_url, self.api_url) def test_url_generation(self): # test we can generate API endpoint URLs self.assertEqual(self.client._build_url('endpoint'), 'http://test-base-url/endpoint') def test_echo(self): # test the 'echo' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.echo('echo test') self.client._make_request.assert_called_with('echo', 'echo test') def test_reverse(self): # test the 'reverse' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.reverse('reverse test') self.client._make_request.assert_called_with( 'reverse', 'reverse test') def test_md5(self): # test the 'md5' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.md5('md5 test') self.client._make_request.assert_called_with('md5', 'md5 test') @patch('requests.post') def test_make_request_call(self, mock_requests_post): # test _make_request actually makes the API call correctly with the given POST data and method data_value = 'an example data value' self.client._make_request('foo', data_value) mock_requests_post.assert_called_with( self.api_url + 'foo', data={'value': data_value}) def test_make_request_return_value(self): # test _make_request returns response content when successful resp = MagicMock() resp.status_code = 200 # unused for now! resp.text = 'return value' with patch('requests.post', return_value=resp) as a: data = self.client._make_request( 'foo', 'hello world') self.assertEqual(data, 'return value') @patch('requests.post', side_effect=HTTPError('eek!')) def test_requests_exception(self, mock_requests_post): # test _make_request propagates Requests errors as EchoClientErrors with self.assertRaises(EchoClientError) as e: self.client._make_request('foo', 'hello world') self.assertTrue( e.exception.message.startswith('HTTPError')) API calls URL Generation Request/ response Exception raising
31. def test_echo(self): # test the 'echo' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.echo('echo test') self.client._make_request.assert_called_with( 'echo', 'echo test') def _make_request(self, endpoint, value): try: response = requests.post( self._build_url(endpoint), data=dict(value=value)) except RequestException as e: raise EchoClientError(e.__class__.__name__) return response.text MagicMock
32. def test_echo(self): # test the 'echo' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.echo('echo test') self.client._make_request.assert_called_with( 'echo', 'echo test') def echo(self, value): return self.MagicMockInstance('echo', value) We can test that _make_request is called with the correct arguments
33. Manually instantiating MagicMock • from mock import MagicMock • Replace methods or attributes with MagicMock instances, when you have direct access to what is being accessed. MagicMock is magic! Callable. Fakes attributes. Etc, etc…
34. isolated functionality • By mocking the entire _make_request method we have isolated it from testing the public echo function. • This means that we can write test cases that are focused on testing small, specific pieces of functionality. • Moreover, we can extend this technique to mock larger monolithic functions (with possible external calls) through repeated applications. So awesome.
35. isolated functionality • We have verified that our echo API function correctly calls _make_request • But, _make_request uses the external requests library to make a HTTP POST call • No sweat, let’s patch that to verify _make_request correctly calls requests.post!
36. import unittest from mock import MagicMock, patch from echoclient import EchoClient, EchoClientError from requests.exceptions import HTTPError class EchoClientTestCase(unittest.TestCase): def setUp(self): self.api_url = 'http://test-base-url/' self.client = EchoClient(self.api_url) def test_base_url(self): # test the base URL is stored correctly self.assertEqual(self.client.base_url, self.api_url) def test_url_generation(self): # test we can generate API endpoint URLs self.assertEqual(self.client._build_url('endpoint'), 'http://test-base-url/endpoint') def test_echo(self): # test the 'echo' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.echo('echo test') self.client._make_request.assert_called_with('echo', 'echo test') def test_reverse(self): # test the 'reverse' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.reverse('reverse test') self.client._make_request.assert_called_with( 'reverse', 'reverse test') def test_md5(self): # test the 'md5' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.md5('md5 test') self.client._make_request.assert_called_with('md5', 'md5 test') @patch('requests.post') def test_make_request_call(self, mock_requests_post): # test _make_request actually makes the API call correctly with the given POST data and method data_value = 'an example data value' self.client._make_request('foo', data_value) mock_requests_post.assert_called_with( self.api_url + 'foo', data={'value': data_value}) def test_make_request_return_value(self): # test _make_request returns response content when successful resp = MagicMock() resp.status_code = 200 # unused for now! resp.text = 'return value' with patch('requests.post', return_value=resp) as a: data = self.client._make_request( 'foo', 'hello world') self.assertEqual(data, 'return value') @patch('requests.post', side_effect=HTTPError('eek!')) def test_requests_exception(self, mock_requests_post): # test _make_request propagates Requests errors as EchoClientErrors with self.assertRaises(EchoClientError) as e: self.client._make_request('foo', 'hello world') self.assertTrue( e.exception.message.startswith('HTTPError')) API calls URL Generation Request/ response Exception raising
37. @patch('requests.post') def test_make_request_call(self, mock_requests_post): # test _make_request actually makes the API call correctly # with the given POST data and method data_value = 'an example data value' self.client._make_request('foo', data_value) mock_requests_post.assert_called_with( self.api_url + 'foo', data={'value': data_value})
38. @patch('requests.post') def test_make_request_call(self, mock_requests_post): # test _make_request actually makes the API call correctly # with the given POST data and method data_value = 'an example data value' self.client._make_request('foo', data_value) mock_requests_post.assert_called_with( self.api_url + 'foo', data={'value': data_value})
39. @patch('requests.post') def test_make_request_call(self, mock_requests_post): # test _make_request actually makes the API call correctly # with the given POST data and method data_value = 'an example data value' self.client._make_request('foo', data_value) mock_requests_post.assert_called_with( self.api_url + 'foo', data={'value': data_value})
40. def _make_request(self, endpoint, value): try: response = MagicMockInstance(...) except RequestException as e: raise EchoClientError(e.__class__.__name__) return response.text @patch('requests.post') def test_make_request_call(self, mock_requests_post): # test _make_request actually makes the API call correctly # with the given POST data and method data_value = 'an example data value' self.client._make_request('foo', data_value) mock_requests_post.assert_called_with( self.api_url + 'foo', data={'value': data_value}) We can be sure that requests.post is called with the correct arguments
41. patch decorator • Mock out stuff you don’t want happening at test time. • For example, network or disk access. • Passes the patched class/method to the function. • Used to replace code you don’t have direct access to. • Like this: from mock import patch @patch('requests.post') @patch.object(SomeClass, '__init__', return_value=None) def test_method(self, mock_obj, mock_rp):
42. def test_make_request_return_value(self): # test _make_request returns response content when successful resp = MagicMock() resp.status_code = 200 # unused for now! resp.text = 'return value' with patch('requests.post', return_value=resp) as a: data = self.client._make_request('foo', 'hello world') self.assertEqual(data, 'return value') def _make_request(self, endpoint, value): try: response = MagicMockInstance(...) except RequestException as e: raise EchoClientError(e.__class__.__name__) return response.text We return another MagicMock object with attributes that we have specified.
43. import unittest from mock import MagicMock, patch from echoclient import EchoClient, EchoClientError from requests.exceptions import HTTPError class EchoClientTestCase(unittest.TestCase): def setUp(self): self.api_url = 'http://test-base-url/' self.client = EchoClient(self.api_url) def test_base_url(self): # test the base URL is stored correctly self.assertEqual(self.client.base_url, self.api_url) def test_url_generation(self): # test we can generate API endpoint URLs self.assertEqual(self.client._build_url('endpoint'), 'http://test-base-url/endpoint') def test_echo(self): # test the 'echo' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.echo('echo test') self.client._make_request.assert_called_with('echo', 'echo test') def test_reverse(self): # test the 'reverse' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.reverse('reverse test') self.client._make_request.assert_called_with( 'reverse', 'reverse test') def test_md5(self): # test the 'md5' method calls _make_request appropriately self.client._make_request = MagicMock() self.client.md5('md5 test') self.client._make_request.assert_called_with('md5', 'md5 test') @patch('requests.post') def test_make_request_call(self, mock_requests_post): # test _make_request actually makes the API call correctly with the given POST data and method data_value = 'an example data value' self.client._make_request('foo', data_value) mock_requests_post.assert_called_with( self.api_url + 'foo', data={'value': data_value}) def test_make_request_return_value(self): # test _make_request returns response content when successful resp = MagicMock() resp.status_code = 200 # unused for now! resp.text = 'return value' with patch('requests.post', return_value=resp) as a: data = self.client._make_request( 'foo', 'hello world') self.assertEqual(data, 'return value') @patch('requests.post', side_effect=HTTPError('eek!')) def test_requests_exception(self, mock_requests_post): # test _make_request propagates Requests errors as EchoClientErrors with self.assertRaises(EchoClientError) as e: self.client._make_request('foo', 'hello world') self.assertTrue( e.exception.message.startswith('HTTPError')) API calls URL Generation Request/ response Exception raising
44. testing for exceptions • requests.post could raise various HTTP errors which is caught and re-raised in _make_request. • But how do we force these errors to happen, so we can test for them? • We can use the side_effect function to raise an exception. (Also useful to provide a custom function/iterable for return values). • Test cases can now be isolated to capture exception handling code only.
45. @patch('requests.post', side_effect=HTTPError('eek!')) def test_requests_exception(self, mock_requests_post): # test _make_request propagates Requests errors as EchoClientErr with self.assertRaises(EchoClientError) as e: self.client._make_request('foo', 'hello world') self.assertTrue(e.exception.message.startswith('HTTPError')) def _make_request(self, endpoint, value): try: response = MagicMockInstance(...) except RequestException as e: raise EchoClientError(e.__class__.__name__) return response.text We create a MagicMock object that will raise an exception when called.
46. EchoClient must work!
47. EchoView EchoClient EchoServer HTTP Python Echo Enterprise Architecture
48. The user visible interface
49. from sys import argv from client.echoclient import EchoClient class EchoView(object): def __init__(self, base_url): self.client = EchoClient(base_url) def echo(self, text): return self.client.echo(text) def reverse(self, text): return self.client.reverse(text) def md5(self, text): return self.client.md5(text) def process_echo_command(base_url, command, arguments): e = EchoView(base_url) text = ' '.join(arguments) return getattr(e, command)(text) if __name__ == '__main__': print process_echo_command(argv[1], argv[2], argv[3:])
50. whoever wrote EchoClient is a genius! His code works exactly as documented.
51. What do we need to test? We know that the EchoClient works as documented. (See rule #1) We just need to make sure we call the EchoClient properly.
52. import unittest from mock import patch, MagicMock from echo_view import EchoView, process_echo_command from client.echoclient import EchoClient class EchoViewTestCase(unittest.TestCase): @patch.object(EchoClient, '__init__', return_value=None) def test_echo_view_init(self, mock_init): # test initialisation of EchoView instance ev = EchoView('base-url') mock_init.assert_called_with('base-url') @patch.object(EchoClient, 'echo') def test_echo_call(self, mock_echo): # test calling the echo method ev = EchoView('base-url') ev.echo('test_value') mock_echo.assert_called_with('test_value') @patch.object(EchoClient, 'reverse') def test_reverse_call(self, mock_reverse): # test calling the reverse method ev = EchoView('base-url') ev.reverse('test_value') mock_reverse.assert_called_with('test_value') @patch.object(EchoClient, 'md5') def test_md5_call(self, mock_md5): # test calling the md5 method ev = EchoView('base-url') ev.md5('test_value') mock_md5.assert_called_with('test_value') def test_process_command_dispatch(self): # test the command-line function dispatch with patch('echo_view.EchoView') as mock_class: mock_obj = mock_class.return_value mock_foo = MagicMock() # attach a 'foo' attribute to test with mock_obj.foo = mock_foo process_echo_command('base-url', 'foo', ['hello', 'world']) mock_foo.assert_called_with('hello world')
53. from client.echoclient import EchoClient @patch.object(EchoClient, 'echo') def test_echo_call(self, mock_echo): # test calling the echo method ev = EchoView('base-url') ev.echo('test_value') mock_echo.assert_called_with('test_value')
54. def process_echo_command(base_url, command, arguments): e = EchoView(base_url) text = ' '.join(arguments) return getattr(e, command)(text) class EchoView(object): ... def md5(self, text): return self.client.md5(text) Dynamically find the appropriate function at runtime using Python magic methods
55. def test_process_command_dispatch(self): # test the command-line function dispatch with patch('echo_view.EchoView') as mock_class: mock_obj = mock_class.return_value mock_foo = MagicMock() # attach a 'foo' attribute to test with mock_obj.foo = mock_foo process_echo_command('base-url', 'foo', ['hello', 'world']) mock_foo.assert_called_with('hello world') We can’t patch __getattr__ directly. Instead we attach a MagicMock attribute to the object instance and test if it is called
56. EchoView must work!
57. EchoView EchoClient EchoServer HTTP Python Echo Enterprise Architecture
58. enterprise echo architecture must work!
59. conclusions • Unit tests should test small, isolated, pieces of functionality. They should execute fast and not rely on other components. • Rely on the assumption that external code and services should work as advertised. Don’t retest them! • Mocking provides a mechanism to decouple application logic with these external services. Use it!
60. python mock library • Thank you Michael Foord! • Patch classes, attributes, with MagicMock instances • Numerous built-in assertion methods • This is just scratching the surface! • http://www.voidspace.org.uk/python/mock/ • Bing search Python Mock (3rd result)
61. the end enjoy testing.