How the Mock library helps me developing client-side applications Cheng-Lung Sung (宋政隆) @clsung 13年9月15日星期日

About Me • Freelancer • semi-active FreeBSD ports committer • Enjoy learning new things • Started coding in Python since three years ago... • http://www.plurk.com/API/ • Python (with mox), also include • Perl, PHP, JS, Go in these years. 13年9月15日星期日

What is Mock? • “mock allows you to replace parts of your system under test with mock/stub/fake objects and make assertions about how they have been used.” 13年9月15日星期日

Python Mock libraries • Chai: https://github.com/agoragames/chai • Last Update: 2013-09-09, v0.4.2 • Flexmock: http://has207.github.io/flexmock/ • Last Update: 2013-03-30, v0.9.7 • MiniMock: http://pypi.python.org/pypi/MiniMock • Last Update: 2013-03-19, v1.2.8 • Mock: http://www.voidspace.org.uk/python/mock/ • Last Update: 2012-11-05, v1.0.1 • Mocker: http://labix.org/mocker • Last Update: 2012-05-25, v1.1.1 • Fudge: http://farmdev.com/projects/fudge/ • Last Update 2011-03-12, v.1.0.3 • Mox: http://code.google.com/p/pymox/ • Last Update: 2010-08-15, v0.5.3 13年9月15日星期日

Which one to choose? http://www.flickr.com/photos/99796131@N00/493788823 13年9月15日星期日

• Tried with Mox, Flexmock • google://“mox flexmock mock python” • However, • “mock is now part of the Python standard library, available as unittest.mock in Python 3.3 onwards.” How I choose mock 13年9月15日星期日

Common usages of Mock library >>> import mock >>> dir(mock) ['...', ..., 'MagicMock', ‘Mock’, ..., ‘patch’, ...] >>> from mock import Mock >>> obj = Mock() >>> obj >>> obj.foo >>> obj.foo.bar 13年9月15日星期日

Mock >>> f = Mock(return_value='foo') >>> f(1, 'a', name='bar') 'foo' >>> f.assert_called_once_with(2, 'a', name='bar') ........ AssertionError: Expected call: mock(2, 'a', name='bar') Actual call: mock(1, 'a', name='bar') >>> f.assert_called_once_with(1, ‘a’, name=‘bar’) >>> f.call_count 1 >>> f.called True >>> f.call_args call(1, 'a', name='bar') 13年9月15日星期日

MagicMock >>> g = MagicMock(return_value='foo') >>> g(1, 'a', name='bar') 'foo' >>> g.assert_called_once_with(1, ‘a’, name=‘bar’) >>> g.call_count 1 >>> g.call_args call(1, 'a', name='bar') >>> g[2] >>> f = Mock(return_value='foo') >>> f[2] TypeError: 'Mock' object does not support indexing 13年9月15日星期日

@patch • Default return: MagicMock() • Example: test folder existence and ... • Code: • if os.path.exists(folder) and \ os.path.isdir(folder): foo = bar • Mock via @decorator: • @patch('os.path.exists', Mock(return_value=True)) @patch('os.path.isdir', Mock(return_value=True)) def test_create_subfolders(self): ... • Mock via context manager: • with patch('os.path.exists', Mock(return_value=True)) as m: os.path.exists('foo.txt') • with patch('os.path.isdir', return_value=True) as mm: os.path.isdir('bar') 13年9月15日星期日

When to use Mock • External • test code that reaches external resources • Expensive • test code which costs lots of time/ components to execute (or setup) • Exception • test code which raises side effects 13年9月15日星期日

How mock helps me... • Develop/test client-side applications • Server-side APIs (Google Drive v2) • Exceptions (yaah~, 500, 400 undocumented) • Response resources • Client-side request library (Requests v1.2) • HTTP requests • HTTP response in Json • File/folder event tests • create/remove/move folder • upload/update/move/remove files 13年9月15日星期日

Google Drive Simulator GUI, wxPython Core Disk I/O Application Overview Lots  of  requests  and  Json  responses Lots  of  exceptions  including   {403  Rate  Limit  Exceeded,   401  Invalid  Access  Token, 412  Precondition  Failed, 500  Internal  Server  Error, ...} Simulate  disk  i/o   event  for  specified   folder  and  reflect  to   remote  storage,  i.e.   Google  Drive. 13年9月15日星期日

Google Drive API v2 • Folder Create/Modify (POST) • File Upload/Update (POST, PUT) • media upload • Resumable upload • Folder/File Remove/Move (UPDATE, PUT) • Authorization • OAuth2 - refresh_token (POST) 13年9月15日星期日

Local I/O modification • Folder Create/Move/Remove event • File Create/Modify/Move/Remove event • Bulk Operation • Folder/File mixed events 13年9月15日星期日

Mock open() - read >>> from mock import mock_open, patch >>> content = [“line1”, “line2”, “line3”, ...] >>> with patch('__builtin__.open', \ mock_open(read_data=content)) as m: with open('a', 'r') as f: print f.read() 13年9月15日星期日

Mock open() - iterating >>> content = [“line1”, “line2”, “line3”, ...] >>> with patch('__builtin__.open') as m: m.return_value = MagicMock() enter = m.return_value.__enter__ enter.return_value.__iter__.return_value = content with open('a', 'r') as f: for line in f: print line 13年9月15日星期日

Mock open() - StringIO >>> strIO = StringIO(["line1", "line2", "line3", ...]) >>> with patch('__builtin__.open') as m: m.return_value = MagicMock() m.return_value.__enter__.return_value = strIO with open('a', 'r') as f: for line in f: print line f.seek(0) for line in f: print line 13年9月15日星期日

patch object in setUp() def setUp(self): patcher = patch.object(requests, \ 'request', \ autospec=True) self.request = patcher.start() self.addCleanup(patcher.stop) # or def setUp(self): patch.object(requests, ‘requests’, autospec=True).start() patch.object(requests, ‘get’, autospect=True).start() def tearDown(self): patch.stopall() # ensure all patched are ‘unpatched’ 13年9月15日星期日

patch HTTP request def setUp(self): patcher = patch.object(requests, 'get', autospec=True) self.get = patcher.start() self.addCleanup(patcher.stop) def test_parse_google_response(self): with patch('requests.Response') as mock_resp: mock_resp.status_code = 200 mock_resp.content = "google's response content" mock_resp.raw = StringIO("google's response raw") self.get.return_value = mock_resp 13年9月15日星期日

patch as @decorator @patch.object(requests.Session, 'request') @patch('requests.Response') def test_parse_google_response(self, mock_resp, mock_ress): mock_resp.status_code = 200 mock_resp.json.return_value = {'id': 'abc'} mock_sess.return_value = mock_resp ... expected = [call('POST', 'https://www.googleapis.com/upload/drive/v2/files'), call('PUT', 'https://www.googleapis.com/drive/v2/files/abc'),] ... mock_sess.assert_has_calls(expected) 13年9月15日星期日

Stub the response class GoogleDriveItemHelper(object): @staticmethod def buildGDItem(parent_item, title, is_folder, is_share=False, description='', md5=''): return { "title": title, "mimeType": ("application/vnd.google-apps.folder" if is_folder else "application/octet-stream"), "parents": [parent_item], "id": randstr(20), .... "downloadUrl": "http://{0}".format(randstr(18)), "md5Checksum": (None if is_folder else md5), "createdDate": datetime.fromtimestamp(time.time()).isoformat(), } # usage: >>> gd_item = GoogleDriveItemHelper.buildGDItem(None, 'title1', ....) 13年9月15日星期日

mock/fake GD item def setUp(self): patcher = patch.object(requests, 'request', autospec=True) self.request = patcher.start() ... def test_create_google_drive_item(self): item1 = GoogleDriveItemHelper.buildGDItem(None, "folder1", is_folder=True) with patch('requests.Response') as mock_resp: mock_resp.status_code = 200 mock_resp.json = Mock(return_value=item1) self.request.return_value = mock_resp 13年9月15日星期日

mock/fake GD items def test_create_and_rename_google_drive_item(self): item1 = GoogleDriveItemHelper.buildGDItem(None, "folder1", is_folder=True) item2 = GoogleDriveItemHelper.buildGDItem(None, "folder2", is_folder=True) def yieldItem(): yield item1 yield item2 yield ... with patch('requests.Response') as mock_resp: mock_resp.status_code = 200 mock_resp.json = Mock(side_effect=yieldItem()) self.request.return_value = mock_resp 13年9月15日星期日

Exceptions def test_api_request(self): with patch('requests.Response') as mock_resp: with self.assertRaises(requests.ConnectionError): mock_resp.status_code = 500 status_code, result = self.gdapi._api_request('POST', '/v2/api') with patch.object(gdapi, 'refresh_access_token', autospec=True) as refresh: refresh.return_value = True with self.assertRaises(gdapi.AccessTokenError): mock_resp.status_code = 401 status_code, result = self.gdapi._api_request('POST', '/v2/api') 13年9月15日星期日

patch @decorator def retry(ExceptionToHandle, tries, delay=3, backoff=2, logger_name=None): def deco_retry(f): def f_retry(*args, **kwargs): while tries >= 1: try: return f(*args, **kwargs) except ExceptionToHandle as e: tries -= 1 return False return wraps(f)(f_retry) # true decorator -> decorated function return deco_retry # usage @retry(requests.ConnectionError, 10, delay=0.5) def api_request(...): 13年9月15日星期日

patch @retry decorator import os import unittest from mock import patch # mock the retry decorator before any module loads it patch('gdapi.utils.retry', lambda x, y, delay: lambda z: z).start() from gdapi.apirequest import APIRequest from gdapi.errors import GoogleApiError import requests class Test_cred_functions(unittest.TestCase): ... 13年9月15日星期日

Check logger output def logAction(self, *args): import inspect self.log.info(u'{0}: {1}'.format(inspect.stack(args[0])[3][3], repr(args[1:]))) return True def create_patch(self, classname, name): patcher = patch.object(classname, name, autospec=True) thing = patcher.start() self.addCleanup(patcher.stop) return thing 13年9月15日星期日

Check logger output def setUp(self, *args): self.mock_move_file = self.create_patch( Simulator, 'move_file') self.mock_move_file.side_effect = self.logAction ... def test_rename_update_file(self): self.move_file('file1.txt', 'file2.txt') self.update_file('file2.txt') with LogCapture('simu') as logcheck: self.assertTrue(self.robot.executeScript()) logcheck.check( ('simu', 'INFO', u"move_file: (u'file1.txt', u'file2.txt')"), ('simu', 'INFO', u"update_file: (u'file2.txt',)"), ) 13年9月15日星期日

Is mock library save my life project? 13年9月15日星期日

When dealing with external 3rd party APIs 13年9月15日星期日

When dealing with external backend errors (fixed by Google) 13年9月15日星期日

Other helpful modules • factory_boy • httpretty • testfixtures 13年9月15日星期日

Thank you https://github.com/clsung/ http://dev.clsung.tw/ 13年9月15日星期日