Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How the Mock library helps me developing client-side applications

How the Mock library helps me developing client-side applications

Slide of my presentation @ PyCon APAC 2013 Japan

9dc1fb93b959c0d838bf6a900306d9b9?s=128

Cheng-Lung Sung

September 15, 2013
Tweet

Transcript

  1. How the Mock library helps me developing client-side applications Cheng-Lung

    Sung (宋政隆) @clsung 13年9月15日星期日
  2. About Me • Freelancer • semi-active FreeBSD ports committer •

    Enjoy learning new things • Started coding in Python since three years ago... • http://www.plurk.com/API/ • Python (with mox), also include • Perl, PHP, JS, Go in these years. 13年9月15日星期日
  3. What is Mock? • “mock allows you to replace parts

    of your system under test with mock/stub/fake objects and make assertions about how they have been used.” 13年9月15日星期日
  4. Python Mock libraries • Chai: https://github.com/agoragames/chai • Last Update: 2013-09-09,

    v0.4.2 • Flexmock: http://has207.github.io/flexmock/ • Last Update: 2013-03-30, v0.9.7 • MiniMock: http://pypi.python.org/pypi/MiniMock • Last Update: 2013-03-19, v1.2.8 • Mock: http://www.voidspace.org.uk/python/mock/ • Last Update: 2012-11-05, v1.0.1 • Mocker: http://labix.org/mocker • Last Update: 2012-05-25, v1.1.1 • Fudge: http://farmdev.com/projects/fudge/ • Last Update 2011-03-12, v.1.0.3 • Mox: http://code.google.com/p/pymox/ • Last Update: 2010-08-15, v0.5.3 13年9月15日星期日
  5. Which one to choose? http://www.flickr.com/photos/99796131@N00/493788823 13年9月15日星期日

  6. • Tried with Mox, Flexmock • google://“mox flexmock mock python”

    • However, • “mock is now part of the Python standard library, available as unittest.mock in Python 3.3 onwards.” How I choose mock 13年9月15日星期日
  7. Common usages of Mock library >>> import mock >>> dir(mock)

    ['...', ..., 'MagicMock', ‘Mock’, ..., ‘patch’, ...] >>> from mock import Mock >>> obj = Mock() >>> obj <Mock id='4339813712'> >>> obj.foo <Mock name='mock.foo' id='4339812240'> >>> obj.foo.bar <Mock name='mock.foo.bar' id='4339813968'> 13年9月15日星期日
  8. Mock >>> f = Mock(return_value='foo') >>> f(1, 'a', name='bar') 'foo'

    >>> f.assert_called_once_with(2, 'a', name='bar') ........ AssertionError: Expected call: mock(2, 'a', name='bar') Actual call: mock(1, 'a', name='bar') >>> f.assert_called_once_with(1, ‘a’, name=‘bar’) >>> f.call_count 1 >>> f.called True >>> f.call_args call(1, 'a', name='bar') 13年9月15日星期日
  9. MagicMock >>> g = MagicMock(return_value='foo') >>> g(1, 'a', name='bar') 'foo'

    >>> g.assert_called_once_with(1, ‘a’, name=‘bar’) >>> g.call_count 1 >>> g.call_args call(1, 'a', name='bar') >>> g[2] <MagicMock name='mock.__getitem__()' id='4512179536'> >>> f = Mock(return_value='foo') >>> f[2] TypeError: 'Mock' object does not support indexing 13年9月15日星期日
  10. @patch • Default return: MagicMock() • Example: test folder existence

    and ... • Code: • if os.path.exists(folder) and \ os.path.isdir(folder): foo = bar • Mock via @decorator: • @patch('os.path.exists', Mock(return_value=True)) @patch('os.path.isdir', Mock(return_value=True)) def test_create_subfolders(self): ... • Mock via context manager: • with patch('os.path.exists', Mock(return_value=True)) as m: os.path.exists('foo.txt') • with patch('os.path.isdir', return_value=True) as mm: os.path.isdir('bar') 13年9月15日星期日
  11. When to use Mock • External • test code that

    reaches external resources • Expensive • test code which costs lots of time/ components to execute (or setup) • Exception • test code which raises side effects 13年9月15日星期日
  12. How mock helps me... • Develop/test client-side applications • Server-side

    APIs (Google Drive v2) • Exceptions (yaah~, 500, 400 undocumented) • Response resources • Client-side request library (Requests v1.2) • HTTP requests • HTTP response in Json • File/folder event tests • create/remove/move folder • upload/update/move/remove files 13年9月15日星期日
  13. Google Drive Simulator GUI, wxPython Core Disk I/O Application Overview

    Lots  of  requests  and  Json  responses Lots  of  exceptions  including   {403  Rate  Limit  Exceeded,   401  Invalid  Access  Token, 412  Precondition  Failed, 500  Internal  Server  Error, ...} Simulate  disk  i/o   event  for  specified   folder  and  reflect  to   remote  storage,  i.e.   Google  Drive. 13年9月15日星期日
  14. Google Drive API v2 • Folder Create/Modify (POST) • File

    Upload/Update (POST, PUT) • media upload • Resumable upload • Folder/File Remove/Move (UPDATE, PUT) • Authorization • OAuth2 - refresh_token (POST) 13年9月15日星期日
  15. Local I/O modification • Folder Create/Move/Remove event • File Create/Modify/Move/Remove

    event • Bulk Operation • Folder/File mixed events 13年9月15日星期日
  16. Mock open() - read >>> from mock import mock_open, patch

    >>> content = [“line1”, “line2”, “line3”, ...] >>> with patch('__builtin__.open', \ mock_open(read_data=content)) as m: with open('a', 'r') as f: print f.read() 13年9月15日星期日
  17. Mock open() - iterating >>> content = [“line1”, “line2”, “line3”,

    ...] >>> with patch('__builtin__.open') as m: m.return_value = MagicMock() enter = m.return_value.__enter__ enter.return_value.__iter__.return_value = content with open('a', 'r') as f: for line in f: print line 13年9月15日星期日
  18. Mock open() - StringIO >>> strIO = StringIO(["line1", "line2", "line3",

    ...]) >>> with patch('__builtin__.open') as m: m.return_value = MagicMock() m.return_value.__enter__.return_value = strIO with open('a', 'r') as f: for line in f: print line f.seek(0) for line in f: print line 13年9月15日星期日
  19. patch object in setUp() def setUp(self): patcher = patch.object(requests, \

    'request', \ autospec=True) self.request = patcher.start() self.addCleanup(patcher.stop) # or def setUp(self): patch.object(requests, ‘requests’, autospec=True).start() patch.object(requests, ‘get’, autospect=True).start() def tearDown(self): patch.stopall() # ensure all patched are ‘unpatched’ 13年9月15日星期日
  20. patch HTTP request def setUp(self): patcher = patch.object(requests, 'get', autospec=True)

    self.get = patcher.start() self.addCleanup(patcher.stop) def test_parse_google_response(self): with patch('requests.Response') as mock_resp: mock_resp.status_code = 200 mock_resp.content = "google's response content" mock_resp.raw = StringIO("google's response raw") self.get.return_value = mock_resp 13年9月15日星期日
  21. patch as @decorator @patch.object(requests.Session, 'request') @patch('requests.Response') def test_parse_google_response(self, mock_resp, mock_ress):

    mock_resp.status_code = 200 mock_resp.json.return_value = {'id': 'abc'} mock_sess.return_value = mock_resp ... expected = [call('POST', 'https://www.googleapis.com/upload/drive/v2/files'), call('PUT', 'https://www.googleapis.com/drive/v2/files/abc'),] ... mock_sess.assert_has_calls(expected) 13年9月15日星期日
  22. Stub the response class GoogleDriveItemHelper(object): @staticmethod def buildGDItem(parent_item, title, is_folder,

    is_share=False, description='', md5=''): return { "title": title, "mimeType": ("application/vnd.google-apps.folder" if is_folder else "application/octet-stream"), "parents": [parent_item], "id": randstr(20), .... "downloadUrl": "http://{0}".format(randstr(18)), "md5Checksum": (None if is_folder else md5), "createdDate": datetime.fromtimestamp(time.time()).isoformat(), } # usage: >>> gd_item = GoogleDriveItemHelper.buildGDItem(None, 'title1', ....) 13年9月15日星期日
  23. mock/fake GD item def setUp(self): patcher = patch.object(requests, 'request', autospec=True)

    self.request = patcher.start() ... def test_create_google_drive_item(self): item1 = GoogleDriveItemHelper.buildGDItem(None, "folder1", is_folder=True) with patch('requests.Response') as mock_resp: mock_resp.status_code = 200 mock_resp.json = Mock(return_value=item1) self.request.return_value = mock_resp 13年9月15日星期日
  24. mock/fake GD items def test_create_and_rename_google_drive_item(self): item1 = GoogleDriveItemHelper.buildGDItem(None, "folder1", is_folder=True)

    item2 = GoogleDriveItemHelper.buildGDItem(None, "folder2", is_folder=True) def yieldItem(): yield item1 yield item2 yield ... with patch('requests.Response') as mock_resp: mock_resp.status_code = 200 mock_resp.json = Mock(side_effect=yieldItem()) self.request.return_value = mock_resp 13年9月15日星期日
  25. Exceptions def test_api_request(self): with patch('requests.Response') as mock_resp: with self.assertRaises(requests.ConnectionError): mock_resp.status_code

    = 500 status_code, result = self.gdapi._api_request('POST', '/v2/api') with patch.object(gdapi, 'refresh_access_token', autospec=True) as refresh: refresh.return_value = True with self.assertRaises(gdapi.AccessTokenError): mock_resp.status_code = 401 status_code, result = self.gdapi._api_request('POST', '/v2/api') 13年9月15日星期日
  26. patch @decorator def retry(ExceptionToHandle, tries, delay=3, backoff=2, logger_name=None): def deco_retry(f):

    def f_retry(*args, **kwargs): while tries >= 1: try: return f(*args, **kwargs) except ExceptionToHandle as e: tries -= 1 return False return wraps(f)(f_retry) # true decorator -> decorated function return deco_retry # usage @retry(requests.ConnectionError, 10, delay=0.5) def api_request(...): 13年9月15日星期日
  27. patch @retry decorator import os import unittest from mock import

    patch # mock the retry decorator before any module loads it patch('gdapi.utils.retry', lambda x, y, delay: lambda z: z).start() from gdapi.apirequest import APIRequest from gdapi.errors import GoogleApiError import requests class Test_cred_functions(unittest.TestCase): ... 13年9月15日星期日
  28. Check logger output def logAction(self, *args): import inspect self.log.info(u'{0}: {1}'.format(inspect.stack(args[0])[3][3],

    repr(args[1:]))) return True def create_patch(self, classname, name): patcher = patch.object(classname, name, autospec=True) thing = patcher.start() self.addCleanup(patcher.stop) return thing 13年9月15日星期日
  29. Check logger output def setUp(self, *args): self.mock_move_file = self.create_patch( Simulator,

    'move_file') self.mock_move_file.side_effect = self.logAction ... def test_rename_update_file(self): self.move_file('file1.txt', 'file2.txt') self.update_file('file2.txt') with LogCapture('simu') as logcheck: self.assertTrue(self.robot.executeScript()) logcheck.check( ('simu', 'INFO', u"move_file: (u'file1.txt', u'file2.txt')"), ('simu', 'INFO', u"update_file: (u'file2.txt',)"), ) 13年9月15日星期日
  30. Is mock library save my life project? 13年9月15日星期日

  31. When dealing with external 3rd party APIs 13年9月15日星期日

  32. When dealing with external backend errors (fixed by Google) 13年9月15日星期日

  33. Other helpful modules • factory_boy • httpretty • testfixtures 13年9月15日星期日

  34. Thank you https://github.com/clsung/ http://dev.clsung.tw/ 13年9月15日星期日